00000100:00100000:0.0:1789760678.864164:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4800 00000100:00100000:0.0:1789760678.864176:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 689111972581 time 689165479316 state 0x2 ref 0 00000100:00100000:0.0:1789760678.871521:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00002000:00010000:0.0:1789760678.902199:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19542016 -> 19546112 00010000:00010000:0.0:1789760678.902215:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0389f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760678.902229:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0389f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760678.902239:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0389f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760678.902251:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760678.902256:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0389f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760678.902266:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0389f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760678.902285:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0389f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760678.902293:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0389f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760678.902434:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493588352:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30643us (30908us total) trans 4294972090 rc 0/0 00000100:00100000:0.0:1789760678.902446:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4801 00000100:00100000:0.0:1789760678.902457:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 689165739732 time 689212800768 state 0x2 ref 0 00002000:00010000:0.0:1789760678.944576:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19546112 -> 19550208 00010000:00010000:0.0:1789760678.944590:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e038a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760678.944604:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e038a6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760678.944612:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e038a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760678.944620:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760678.944625:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e038a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760678.944634:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e038a6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760678.944654:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e038a6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760678.944662:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e038a6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760678.944857:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493588480:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38924us (39248us total) trans 4294972091 rc 0/0 00000100:00100000:0.0:1789760678.944868:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4802 00000100:00100000:0.0:1789760678.944877:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 689213061184 time 689246954137 state 0x2 ref 0 00000100:00100000:0.0:1789760678.948943:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493588608 00000100:00100000:0.0:1789760678.949097:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493588608 00000100:00100000:0.0:1789760678.949151:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 689213061184 time 689246954137 state 0x0 ref 1 00000100:00100000:0.0:1789760678.949166:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 689247214553 time 689246954137 state 0x1 ref 1 00000100:00100000:0.0:1789760678.949227:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 689247214553 time 689290262432 state 0x0 ref 1 00000100:00100000:0.0:1789760678.949239:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7aa7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493588608:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760678.949285:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 19550208, mtime: 0x6aad94a6, atime: 0x0, ctime: 0x6aad94a6, blocks: 0x9807 00010000:00010000:0.0:1789760678.949294:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e038ad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760678.949309:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=689s v=(0 0) 00010000:00010000:0.0:1789760678.949313:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760678.949319:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e038ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760678.949331:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e038ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760679.047708:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19558400 -> 19562496 00010000:00010000:0.0:1789760679.047721:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e038bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19558400->19562495] (req 19558400->19562495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760679.047730:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e038bb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19558400->19562495] (req 19558400->19562495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760679.047739:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e038bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19558400->19562495] (req 19558400->19562495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760679.047746:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760679.047751:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e038bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19558400->19562495] (req 19558400->19562495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760679.047760:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e038bb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19558400->19562495] (req 19558400->19562495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760679.047781:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e038bb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19558400->19562495] (req 19558400->19562495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760679.047789:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e038bb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19558400->19562495] (req 19558400->19562495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760679.047910:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493588864:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36211us (36599us total) trans 4294972094 rc 0/0 00000100:00100000:0.0:1789760679.047922:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4805 00000100:00100000:0.0:1789760679.047930:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 689323111514 time 689352719874 state 0x2 ref 0 00000100:00100000:0.0:1789760679.100515:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493589120 00000100:00100000:0.0:1789760679.100527:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493589120 00000100:00100000:0.0:1789760679.100588:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 689352980290 time 689393507720 state 0x0 ref 1 00000100:00100000:0.0:1789760679.100604:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 689393768136 time 689393507720 state 0x1 ref 1 00000100:00100000:0.0:1789760679.100640:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 689393768136 time 689441674641 state 0x0 ref 1 00000100:00100000:0.0:1789760679.100652:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493589120:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760679.100702:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 19566592, mtime: 0x6aad94a6, atime: 0x0, ctime: 0x6aad94a6, blocks: 0x9807 00010000:00010000:0.0:1789760679.100710:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e038c9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760679.100726:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=689s v=(0 0) 00010000:00010000:0.0:1789760679.100731:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760679.100737:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e038c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760679.100751:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e038c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760679.360388:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493589632 00000100:00100000:0.0:1789760679.360403:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493589632 00000100:00100000:0.0:1789760679.360542:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 689553192994 time 689634650737 state 0x0 ref 1 00000100:00100000:0.0:1789760679.360559:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 689634911153 time 689634650737 state 0x1 ref 1 00000100:00100000:0.0:1789760679.360599:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 689634911153 time 689701634024 state 0x0 ref 1 00000100:00100000:0.0:1789760679.360612:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620f39f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493589632:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760679.360664:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 19582976, mtime: 0x6aad94a6, atime: 0x0, ctime: 0x6aad94a6, blocks: 0x9807 00010000:00010000:0.0:1789760679.360673:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e038e5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760679.360689:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=689s v=(0 0) 00010000:00010000:0.0:1789760679.360693:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760679.360700:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e038e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760679.360713:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e038e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760679.773336:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493590400 00000100:00100000:0.0:1789760679.773345:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493590400 00000100:00100000:0.0:1789760679.773392:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 689924121461 time 690034417441 state 0x0 ref 1 00000100:00100000:0.0:1789760679.773407:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 690034677857 time 690034417441 state 0x1 ref 1 00000100:00100000:0.0:1789760679.773436:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 690034677857 time 690114472353 state 0x0 ref 1 00000100:00100000:0.0:1789760679.773451:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493590400:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760679.773565:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 19607552, mtime: 0x6aad94a7, atime: 0x0, ctime: 0x6aad94a7, blocks: 0x9807 00010000:00010000:0.0:1789760679.773573:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0390f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760679.773589:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1789760679.773593:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760679.773600:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0390f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760679.773612:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0390f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760679.953652:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19615744 -> 19619840 00010000:00010000:0.0:1789760679.953667:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0391d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760679.953681:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0391d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760679.953689:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0391d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760679.953696:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760679.953700:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0391d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760679.953710:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0391d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760679.953728:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0391d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760679.953735:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0391d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760679.953826:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493590656:12345-192.168.203.60@tcp:4:dd.0 Request processed in 86030us (86260us total) trans 4294972108 rc 0/0 00000100:00100000:0.0:1789760679.953838:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4819 00000100:00100000:0.0:1789760679.953848:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 690166135868 time 690208815865 state 0x2 ref 0 00000100:00100000:0.0:1789760679.956374:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760680.028192:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493590912 00000100:00100000:0.0:1789760680.028203:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493590912 00000100:00100000:0.0:1789760680.028256:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 690209076281 time 690297610334 state 0x0 ref 1 00000100:00100000:0.0:1789760680.028271:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 690297870750 time 690297610334 state 0x1 ref 1 00000100:00100000:0.0:1789760680.028303:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 690297870750 time 690369338418 state 0x0 ref 1 00000100:00100000:0.0:1789760680.028315:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493590912:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760680.028364:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 19623936, mtime: 0x6aad94a7, atime: 0x0, ctime: 0x6aad94a7, blocks: 0x9807 00010000:00010000:0.0:1789760680.028373:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0392b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.028389:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1789760680.028393:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760680.028399:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0392b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.028413:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0392b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760680.132174:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493591168 00000100:00100000:0.0:1789760680.132185:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493591168 00000100:00100000:0.0:1789760680.132239:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 690369598834 time 690424958675 state 0x0 ref 1 00000100:00100000:0.0:1789760680.132256:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 690425219091 time 690424958675 state 0x1 ref 1 00000100:00100000:0.0:1789760680.132285:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 690425219091 time 690473320836 state 0x0 ref 1 00000100:00100000:0.0:1789760680.132297:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493591168:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760680.132343:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 19632128, mtime: 0x6aad94a7, atime: 0x0, ctime: 0x6aad94a7, blocks: 0x9807 00010000:00010000:0.0:1789760680.132352:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e03939 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.132367:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1789760680.132372:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760680.132378:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e03939 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.132393:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e03939 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760680.166114:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972112 is committed 00002000:00010000:0.0:1789760680.246705:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19640320 -> 19644416 00010000:00010000:0.0:1789760680.246720:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e03947 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.246733:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e03947 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.246742:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e03947 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.246750:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760680.246755:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e03947 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.246764:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e03947 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.246784:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e03947 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.246792:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e03947 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760680.246919:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4361880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493591424:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32487us (32705us total) trans 4294972114 rc 0/0 00000100:00100000:0.0:1789760680.246930:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4825 00000100:00100000:0.0:1789760680.246950:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 690516984002 time 690555452558 state 0x2 ref 0 00000100:00100000:0.0:1789760680.251241:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:0.0:1789760680.280260:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972115 is committed 00000100:00100000:0.0:1789760680.284700:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493591680 00000100:00100000:0.0:1789760680.284709:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493591680 00000100:00100000:0.0:1789760680.284759:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 690555712974 time 690592638071 state 0x0 ref 1 00000100:00100000:0.0:1789760680.284773:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 690592898487 time 690592638071 state 0x1 ref 1 00000100:00100000:0.0:1789760680.284828:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 690592898487 time 690625863164 state 0x0 ref 1 00000100:00100000:0.0:1789760680.284839:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8556a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493591680:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760680.284885:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 19648512, mtime: 0x6aad94a7, atime: 0x0, ctime: 0x6aad94a7, blocks: 0x9807 00010000:00010000:0.0:1789760680.284894:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e03955 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.284911:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1789760680.284915:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760680.284921:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e03955 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.284950:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e03955 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760680.371534:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972117 is committed 00002000:00010000:0.0:1789760680.446644:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19656704 -> 19660800 00010000:00010000:0.0:1789760680.446659:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e03963 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.446671:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e03963 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.446682:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e03963 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.446689:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760680.446694:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e03963 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.446703:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e03963 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.446722:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e03963 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.446730:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e03963 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760680.446816:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493591936:12345-192.168.203.60@tcp:4:dd.0 Request processed in 69431us (69769us total) trans 4294972118 rc 0/0 00000100:00100000:0.0:1789760680.446827:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4829 00000100:00100000:0.0:1789760680.446838:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 690666520769 time 690718407197 state 0x2 ref 0 00002000:00010000:0.0:1789760680.484751:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19660800 -> 19664896 00010000:00010000:0.0:1789760680.484767:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0396a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.484779:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0396a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.484787:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0396a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.484793:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760680.484797:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0396a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.484805:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0396a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.484823:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0396a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.484835:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0396a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760680.484922:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493592064:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34698us (34905us total) trans 4294972119 rc 0/0 00000100:00100000:0.0:1789760680.485577:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4830 00000100:00100000:0.0:1789760680.485592:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 690718667613 time 690791244025 state 0x2 ref 0 00000100:00100000:0.0:1789760680.490322:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493592192 00000100:00100000:0.0:1789760680.490332:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493592192 00000100:00100000:0.0:1789760680.490392:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 690718667613 time 690791244025 state 0x0 ref 1 00000100:00100000:0.0:1789760680.490407:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 690791504441 time 690791244025 state 0x1 ref 1 00000100:00100000:0.0:1789760680.490548:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 690791504441 time 690831582577 state 0x0 ref 1 00000100:00100000:0.0:1789760680.490562:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8556300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493592192:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760680.490609:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 19664896, mtime: 0x6aad94a7, atime: 0x0, ctime: 0x6aad94a7, blocks: 0x9807 00010000:00010000:0.0:1789760680.490618:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03971 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.490633:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1789760680.490637:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760680.490643:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03971 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.490661:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03971 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760680.520399:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19664896 -> 19668992 00010000:00010000:0.0:1789760680.520414:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03971 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.520428:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03971 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.520437:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03971 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.520444:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760680.520448:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03971 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.520457:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03971 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.520477:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03971 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.520485:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03971 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760680.520583:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8556300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493592192:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30018us (30913us total) trans 4294972120 rc 0/0 00000100:00100000:0.0:1789760680.520593:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4831 00000100:00100000:0.0:1789760680.520603:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 690791504441 time 690831582577 state 0x2 ref 0 00002000:00010000:0.0:1789760680.598455:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19673088 -> 19677184 00010000:00010000:0.0:1789760680.598471:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0397f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.598484:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0397f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.598494:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0397f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.598501:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760680.598518:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0397f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.598529:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0397f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.598550:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0397f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.598564:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0397f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760680.598656:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8557b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493592448:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33627us (34049us total) trans 4294972122 rc 0/0 00000100:00100000:0.0:1789760680.598666:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4833 00000100:00100000:0.0:1789760680.598676:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 690865392195 time 690906047806 state 0x2 ref 0 00000100:00100000:0.0:1789760680.602579:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00002000:00010000:0.0:1789760680.693840:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19681280 -> 19685376 00010000:00010000:0.0:1789760680.693855:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0398d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.693868:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0398d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.693877:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0398d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.693884:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760680.693889:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0398d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.693898:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0398d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.693926:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0398d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.693948:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0398d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760680.695284:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493592704:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51892us (52082us total) trans 4294972124 rc 0/0 00000100:00100000:0.0:1789760680.695296:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4835 00000100:00100000:0.0:1789760680.695306:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 690944146615 time 690984407119 state 0x2 ref 0 00000100:00100000:0.0:1789760680.753744:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493592960 00000100:00100000:0.0:1789760680.753755:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493592960 00000100:00100000:0.0:1789760680.753812:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 690984667535 time 691050826963 state 0x0 ref 1 00000100:00100000:0.0:1789760680.753829:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 691051087379 time 691050826963 state 0x1 ref 1 00000100:00100000:0.0:1789760680.753857:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 691051087379 time 691094892502 state 0x0 ref 1 00000100:00100000:0.0:1789760680.753870:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8555180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493592960:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760680.753925:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 19689472, mtime: 0x6aad94a8, atime: 0x0, ctime: 0x6aad94a8, blocks: 0x9807 00010000:00010000:0.0:1789760680.753946:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e0399b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.754054:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=691s v=(0 0) 00010000:00010000:0.0:1789760680.754060:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760680.754067:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e0399b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.754083:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e0399b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760680.835896:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00002000:00010000:0.0:1789760680.892281:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19693568 -> 19697664 00010000:00010000:0.0:1789760680.892297:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e039a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.892310:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e039a2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.892320:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e039a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.892327:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760680.892332:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e039a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.892340:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e039a2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.892359:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e039a2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760680.892371:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e039a2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760680.892669:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493593088:12345-192.168.203.60@tcp:4:dd.0 Request processed in 56285us (56780us total) trans 4294972127 rc 0/0 00000100:00100000:0.0:1789760680.892682:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4838 00000100:00100000:0.0:1789760680.892692:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 691095152918 time 691177401166 state 0x2 ref 0 00000001:00080000:0.0:1789760681.007399:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972129 is committed 00000100:00100000:0.0:1789760681.012471:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760681.065206:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493593600 00000100:00100000:0.0:1789760681.065218:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493593600 00000100:00100000:0.0:1789760681.065288:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 375 00000100:00100000:0.0:1789760681.065299:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7bfe050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2369:x1876699493593600:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:0.0:1789760681.065381:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7bfe050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2369:x1876699493593600:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 84us (291us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760681.065389:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 375 00000100:00100000:0.0:1789760681.066444:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493593984 00000100:00100000:0.0:1789760681.066452:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493593984 00000100:00100000:0.0:1789760681.066499:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 691309182275 time 691355331011 state 0x0 ref 1 00000100:00100000:0.0:1789760681.066515:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 691355591427 time 691355331011 state 0x1 ref 1 00000100:00100000:0.0:1789760681.066540:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 691355591427 time 691407576169 state 0x0 ref 1 00000100:00100000:0.0:1789760681.066555:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fadb7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493593984:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760681.066604:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 19709952, mtime: 0x6aad94a8, atime: 0x0, ctime: 0x6aad94a8, blocks: 0x9807 00010000:00010000:0.0:1789760681.066614:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e039be lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760681.066631:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=691s v=(0 0) 00010000:00010000:0.0:1789760681.066636:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760681.066643:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e039be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760681.066659:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e039be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760681.116835:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19709952 -> 19714048 00010000:00010000:0.0:1789760681.116851:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e039be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760681.116865:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e039be lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760681.116873:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e039be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760681.116881:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760681.116886:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e039be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760681.116895:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e039be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760681.116913:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e039be lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760681.116920:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e039be lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760681.117236:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fadb7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493593984:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50678us (50832us total) trans 4294972131 rc 0/0 00000100:00100000:0.0:1789760681.117246:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4842 00000100:00100000:0.0:1789760681.117256:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 691355591427 time 691407576169 state 0x2 ref 0 00000100:00100000:0.0:1789760681.196608:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:0.0:1789760681.237344:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972134 is committed 00000100:00100000:0.0:1789760681.243005:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760681.296230:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760681.354528:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760681.431572:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760681.437562:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 691637782492 time 691695770264 state 0x0 ref 1 00000100:00100000:0.0:1789760681.437579:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 691696030680 time 691695770264 state 0x1 ref 1 00000100:00100000:0.0:1789760681.438291:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 691696030680 time 691779322017 state 0x0 ref 1 00000100:00100000:0.0:1789760681.438315:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c3097480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493594880:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760681.438372:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 19738624, mtime: 0x6aad94a8, atime: 0x0, ctime: 0x6aad94a8, blocks: 0x9807 00010000:00010000:0.0:1789760681.438380:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e039ef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760681.438396:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=691s v=(0 0) 00010000:00010000:0.0:1789760681.438400:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760681.440722:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e039ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760681.440751:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e039ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760681.543023:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19742720 -> 19746816 00010000:00010000:0.0:1789760681.543037:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e039f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760681.543049:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e039f6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760681.543058:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e039f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760681.543065:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760681.543070:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e039f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760681.543080:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e039f6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760681.543100:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e039f6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760681.543108:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e039f6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760681.543204:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493595008:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39741us (40370us total) trans 4294972139 rc 0/0 00000100:00100000:0.0:1789760681.543214:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4850 00000100:00100000:0.0:1789760681.543223:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 691779582433 time 691844483314 state 0x2 ref 0 00000100:00100000:0.0:1789760681.746506:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760681.858392:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760681.950765:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760681.989171:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00002000:00010000:0.0:1789760682.110752:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19775488 -> 19779584 00010000:00010000:0.0:1789760682.110773:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03a2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.110786:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03a2e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.110802:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03a2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.110809:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760682.110814:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03a2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.110822:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03a2e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.110842:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03a2e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.110850:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03a2e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760682.110950:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a34e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493596032:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60941us (61332us total) trans 4294972147 rc 0/0 00000100:00100000:0.0:1789760682.111009:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4858 00000100:00100000:0.0:1789760682.111020:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 692330820023 time 692390990921 state 0x2 ref 0 00000100:00100000:0.0:1789760682.123477:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:0.0:1789760682.181930:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972148 is committed 00000100:00100000:0.0:1789760682.198774:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:0.0:1789760682.255626:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972149 is committed 00000001:00080000:0.0:1789760682.308358:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972150 is committed 00000100:00100000:0.0:1789760682.313862:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00002000:00010000:0.0:1789760682.377473:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19791872 -> 19795968 00010000:00010000:0.0:1789760682.377494:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03a4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.377507:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03a4a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.377515:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03a4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.377522:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760682.377527:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03a4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.377536:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03a4a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.377557:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03a4a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.377564:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03a4a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760682.377665:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493596544:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63542us (63807us total) trans 4294972151 rc 0/0 00000100:00100000:0.0:1789760682.377679:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4862 00000100:00100000:0.0:1789760682.377690:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 692601449145 time 692655137961 state 0x2 ref 0 00000001:00080000:0.0:1789760682.377732:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972151 is committed 00000100:00100000:0.0:1789760682.384460:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00002000:00010000:0.0:1789760682.474901:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19800064 -> 19804160 00010000:00010000:0.0:1789760682.474929:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e03a58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.474942:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e03a58 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.474952:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e03a58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.475081:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760682.475087:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e03a58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.475098:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e03a58 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.475116:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e03a58 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.475125:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e03a58 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760682.552290:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493597056 00000100:00100000:0.0:1789760682.552300:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493597056 00000100:00100000:0.0:1789760682.552357:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 692765103723 time 692834290608 state 0x0 ref 1 00000100:00100000:0.0:1789760682.552371:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 692834551024 time 692834290608 state 0x1 ref 1 00000100:00100000:0.0:1789760682.637006:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493597184 00000100:00100000:0.0:1789760682.637016:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493597184 00000100:00100000:0.0:1789760682.637070:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 692834551024 time 692893588601 state 0x0 ref 1 00000100:00100000:0.0:1789760682.637086:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 692893849017 time 692893588601 state 0x1 ref 1 00000100:00100000:0.0:1789760682.637117:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 692893849017 time 692978153236 state 0x0 ref 1 00000100:00100000:0.0:1789760682.637129:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493597184:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760682.637180:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 19812352, mtime: 0x6aad94a9, atime: 0x0, ctime: 0x6aad94a9, blocks: 0x9807 00010000:00010000:0.0:1789760682.637189:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03a6d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.637205:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=692s v=(0 0) 00010000:00010000:0.0:1789760682.637210:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760682.637216:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03a6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.637230:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03a6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760682.677891:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19812352 -> 19816448 00002000:00010000:0.0:1789760682.677900:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760681 -> 1789760682 00002000:00010000:0.0:1789760682.677903:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760681 -> 1789760682 00010000:00010000:0.0:1789760682.677916:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03a6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.677939:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03a6d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.677948:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03a6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.677956:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760682.677985:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03a6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.677995:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03a6d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.678016:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03a6d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.678024:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03a6d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760682.678120:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493597184:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40988us (41239us total) trans 4294972156 rc 0/0 00000100:00100000:0.0:1789760682.678130:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4867 00000100:00100000:0.0:1789760682.678139:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 692893849017 time 692978153236 state 0x2 ref 0 00000100:00100000:0.0:1789760682.686027:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493597312 00000100:00100000:0.0:1789760682.686036:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493597312 00000100:00100000:0.0:1789760682.686089:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 692893849017 time 692978153236 state 0x0 ref 1 00000100:00100000:0.0:1789760682.686105:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 692978413652 time 692978153236 state 0x1 ref 1 00000100:00100000:0.0:1789760682.686205:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 692978413652 time 693027239037 state 0x0 ref 1 00000100:00100000:0.0:1789760682.686219:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493597312:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760682.686270:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 19816448, mtime: 0x6aad94aa, atime: 0x0, ctime: 0x6aad94aa, blocks: 0x9807 00010000:00010000:0.0:1789760682.686280:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03a74 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.686296:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760682.686301:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760682.686308:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03a74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.686322:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03a74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760682.733337:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19816448 -> 19820544 00010000:00010000:0.0:1789760682.733359:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03a74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.733373:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03a74 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.733381:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03a74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.733402:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760682.733408:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03a74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.733417:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03a74 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.733439:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03a74 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.733447:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03a74 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760682.733547:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493597312:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47325us (47711us total) trans 4294972157 rc 0/0 00000100:00100000:0.0:1789760682.733560:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4868 00000100:00100000:0.0:1789760682.733570:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 692978413652 time 693027239037 state 0x2 ref 0 00000100:00100000:0.0:1789760682.865371:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493597696 00000100:00100000:0.0:1789760682.865383:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493597696 00000100:00100000:0.0:1789760682.865435:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693085844129 time 693130483182 state 0x0 ref 1 00000100:00100000:0.0:1789760682.865451:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693130743598 time 693130483182 state 0x1 ref 1 00000100:00100000:0.0:1789760682.865487:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693130743598 time 693206523258 state 0x0 ref 1 00000100:00100000:0.0:1789760682.865501:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9621016d2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493597696:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760682.865549:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 19828736, mtime: 0x6aad94aa, atime: 0x0, ctime: 0x6aad94aa, blocks: 0x9807 00010000:00010000:0.0:1789760682.865558:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03a89 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.865573:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760682.865577:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760682.865582:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03a89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760682.865596:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03a89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760682.874629:0:6691:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519242240 00000100:00100000:0.0:1789760682.874638:0:6691:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519242240 00000100:00100000:0.0:1789760682.874692:0:6691:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 177 00000100:00100000:0.0:1789760682.874702:0:6691:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb453800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519242240:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760682.874737:0:6691:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760682.874790:0:6691:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb453800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519242240:12345-0@lo:13:osp-pre-1-0.0 Request processed in 90us (318us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760682.874797:0:6691:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 177 00000100:00100000:0.0:1789760682.933944:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:0.0:1789760682.972088:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972161 is committed 00000001:00080000:0.0:1789760683.009253:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972162 is committed 00000100:00100000:0.0:1789760683.013279:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493598080 00000100:00100000:0.0:1789760683.013287:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493598080 00000100:00100000:0.0:1789760683.013495:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693275821206 time 693317953162 state 0x0 ref 1 00000100:00100000:0.0:1789760683.013509:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693318213578 time 693317953162 state 0x1 ref 1 00000100:00100000:0.0:1789760683.013586:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693318213578 time 693354620855 state 0x0 ref 1 00000100:00100000:0.0:1789760683.013599:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d89880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493598080:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760683.013643:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 19841024, mtime: 0x6aad94aa, atime: 0x0, ctime: 0x6aad94aa, blocks: 0x9807 00010000:00010000:0.0:1789760683.013651:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03a9e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.013667:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760683.013672:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760683.013678:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03a9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.013697:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03a9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760683.041718:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19841024 -> 19845120 00010000:00010000:0.0:1789760683.041736:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03a9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.041749:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03a9e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.041758:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03a9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.041765:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760683.041771:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03a9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.041780:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03a9e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.041799:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03a9e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.041807:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03a9e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760683.041900:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d89880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493598080:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28298us (28721us total) trans 4294972163 rc 0/0 00000100:00100000:0.0:1789760683.041909:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4874 00000100:00100000:0.0:1789760683.041954:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693318213578 time 693354620855 state 0x2 ref 0 00000100:00100000:0.0:1789760683.046522:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760683.141201:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493598464 00000100:00100000:0.0:1789760683.141210:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493598464 00000100:00100000:0.0:1789760683.141267:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693388011135 time 693431832208 state 0x0 ref 1 00000100:00100000:0.0:1789760683.141282:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693432092624 time 693431832208 state 0x1 ref 1 00000100:00100000:0.0:1789760683.141307:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693432092624 time 693482342354 state 0x0 ref 1 00000100:00100000:0.0:1789760683.141319:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493598464:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760683.141367:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 19853312, mtime: 0x6aad94aa, atime: 0x0, ctime: 0x6aad94aa, blocks: 0x9807 00010000:00010000:0.0:1789760683.141375:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e03ab3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.141390:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760683.141460:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760683.141468:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e03ab3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.141484:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e03ab3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760683.172583:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00002000:00010000:0.0:1789760683.262336:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19861504 -> 19865600 00010000:00010000:0.0:1789760683.262353:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03ac1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.262365:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03ac1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.262374:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03ac1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.262381:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760683.262386:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03ac1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.262395:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03ac1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.262414:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03ac1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.262421:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03ac1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760683.262515:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493598720:12345-192.168.203.60@tcp:4:dd.0 Request processed in 61117us (61360us total) trans 4294972168 rc 0/0 00000100:00100000:0.0:1789760683.262525:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4879 00000100:00100000:0.0:1789760683.262534:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693514149178 time 693542414461 state 0x2 ref 0 00000001:00080000:0.0:1789760683.310870:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972169 is committed 00000100:00100000:0.0:1789760683.316227:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493598976 00000100:00100000:0.0:1789760683.316300:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493598976 00000100:00100000:0.0:1789760683.316355:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693542674877 time 693607801108 state 0x0 ref 1 00000100:00100000:0.0:1789760683.316370:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693608061524 time 693607801108 state 0x1 ref 1 00000100:00100000:0.0:1789760683.316497:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693608061524 time 693657531622 state 0x0 ref 1 00000100:00100000:0.0:1789760683.316511:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d08380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493598976:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760683.316564:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 19869696, mtime: 0x6aad94aa, atime: 0x0, ctime: 0x6aad94aa, blocks: 0x9807 00010000:00010000:0.0:1789760683.316583:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03acf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.316599:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760683.316604:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760683.316610:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03acf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.316625:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03acf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760683.393239:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19869696 -> 19873792 00010000:00010000:0.0:1789760683.393254:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03acf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.393267:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03acf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.393277:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03acf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.393285:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760683.393291:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03acf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.393300:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03acf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.393319:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03acf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.393328:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03acf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760683.393428:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d08380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493598976:12345-192.168.203.60@tcp:4:dd.0 Request processed in 76915us (77521us total) trans 4294972170 rc 0/0 00000100:00100000:0.0:1789760683.393439:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4881 00000100:00100000:0.0:1789760683.393449:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693608061524 time 693657531622 state 0x2 ref 0 00000100:00100000:0.0:1789760683.396757:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493599104 00000100:00100000:0.0:1789760683.396767:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493599104 00000100:00100000:0.0:1789760683.396821:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693608061524 time 693657531622 state 0x0 ref 1 00000100:00100000:0.0:1789760683.396836:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693657792038 time 693657531622 state 0x1 ref 1 00000100:00100000:0.0:1789760683.396927:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693657792038 time 693737961853 state 0x0 ref 1 00000100:00100000:0.0:1789760683.396941:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493599104:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760683.397024:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 19873792, mtime: 0x6aad94aa, atime: 0x0, ctime: 0x6aad94aa, blocks: 0x9807 00010000:00010000:0.0:1789760683.397034:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03ad6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.397048:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760683.397053:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760683.397059:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03ad6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.397072:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03ad6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760683.433705:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19873792 -> 19877888 00010000:00010000:0.0:1789760683.433722:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03ad6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.433736:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03ad6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.433747:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03ad6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.433755:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760683.433761:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03ad6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.433770:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03ad6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.433790:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03ad6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.433798:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03ad6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760683.433950:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493599104:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37007us (37280us total) trans 4294972171 rc 0/0 00000100:00100000:0.0:1789760683.434015:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4882 00000100:00100000:0.0:1789760683.434025:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693657792038 time 693737961853 state 0x2 ref 0 00000100:00100000:0.0:1789760683.443490:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493599232 00000100:00100000:0.0:1789760683.443502:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493599232 00000100:00100000:0.0:1789760683.443563:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693657792038 time 693737961853 state 0x0 ref 1 00000100:00100000:0.0:1789760683.443579:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693738222269 time 693737961853 state 0x1 ref 1 00000100:00100000:0.0:1789760683.443612:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693738222269 time 693784647437 state 0x0 ref 1 00000100:00100000:0.0:1789760683.443624:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493599232:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760683.443673:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 19877888, mtime: 0x6aad94aa, atime: 0x0, ctime: 0x6aad94aa, blocks: 0x9807 00010000:00010000:0.0:1789760683.443682:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03add lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.443698:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760683.443702:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760683.443708:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03add lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.443722:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03add lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760683.568001:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19881984 -> 19886080 00010000:00010000:0.0:1789760683.568016:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e03ae4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.568029:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e03ae4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.568037:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e03ae4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.568044:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760683.568049:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e03ae4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.568059:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e03ae4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.568077:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e03ae4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.568084:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e03ae4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760683.568168:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493599360:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42671us (42931us total) trans 4294972173 rc 0/0 00000100:00100000:0.0:1789760683.568178:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4884 00000100:00100000:0.0:1789760683.568187:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693784907853 time 693866517239 state 0x2 ref 0 00000100:00100000:0.0:1789760683.571896:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493599488 00000100:00100000:0.0:1789760683.571905:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493599488 00000100:00100000:0.0:1789760683.571994:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693784907853 time 693866517239 state 0x0 ref 1 00000100:00100000:0.0:1789760683.572010:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693866777655 time 693866517239 state 0x1 ref 1 00000100:00100000:0.0:1789760683.572088:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693866777655 time 693913122203 state 0x0 ref 1 00000100:00100000:0.0:1789760683.572102:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d89500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493599488:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760683.572149:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 19886080, mtime: 0x6aad94aa, atime: 0x0, ctime: 0x6aad94aa, blocks: 0x9807 00010000:00010000:0.0:1789760683.572157:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03aeb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.572171:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760683.572176:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760683.572181:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03aeb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.572193:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03aeb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760683.642687:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19886080 -> 19890176 00002000:00010000:0.0:1789760683.642697:0:13038:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760682 -> 1789760683 00002000:00010000:0.0:1789760683.642700:0:13038:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760682 -> 1789760683 00010000:00010000:0.0:1789760683.642713:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03aeb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.642726:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03aeb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.642735:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03aeb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.642742:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760683.642746:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03aeb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.642761:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03aeb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.642782:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03aeb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.642790:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03aeb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760683.642891:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d89500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493599488:12345-192.168.203.60@tcp:4:dd.0 Request processed in 70787us (71088us total) trans 4294972174 rc 0/0 00000100:00100000:0.0:1789760683.642906:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4885 00000100:00100000:0.0:1789760683.642925:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693866777655 time 693913122203 state 0x2 ref 0 00000100:00100000:0.0:1789760683.651265:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493599616 00000100:00100000:0.0:1789760683.651276:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493599616 00000100:00100000:0.0:1789760683.651327:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693866777655 time 693913122203 state 0x0 ref 1 00000100:00100000:0.0:1789760683.651341:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693913382619 time 693913122203 state 0x1 ref 1 00000100:00100000:0.0:1789760683.651382:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693913382619 time 693992418202 state 0x0 ref 1 00000100:00100000:0.0:1789760683.651402:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493599616:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760683.651449:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 19890176, mtime: 0x6aad94ab, atime: 0x0, ctime: 0x6aad94ab, blocks: 0x9807 00010000:00010000:0.0:1789760683.651457:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03af2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.651473:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760683.651477:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760683.651483:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03af2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.651498:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03af2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760683.715375:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19890176 -> 19894272 00010000:00010000:0.0:1789760683.715392:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03af2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.715406:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03af2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.715414:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03af2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.715422:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760683.715427:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03af2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.715437:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03af2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.715459:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03af2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.715466:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03af2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760683.715559:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493599616:12345-192.168.203.60@tcp:4:dd.0 Request processed in 64161us (64455us total) trans 4294972175 rc 0/0 00000100:00100000:0.0:1789760683.715578:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4886 00000100:00100000:0.0:1789760683.715588:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693913382619 time 693992418202 state 0x2 ref 0 00000100:00100000:0.0:1789760683.782324:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493599872 00000100:00100000:0.0:1789760683.782332:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493599872 00000100:00100000:0.0:1789760683.782377:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 693992678618 time 694073778975 state 0x0 ref 1 00000100:00100000:0.0:1789760683.782388:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 694074039391 time 694073778975 state 0x1 ref 1 00000100:00100000:0.0:1789760683.782410:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 694074039391 time 694123447155 state 0x0 ref 1 00000100:00100000:0.0:1789760683.782419:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d89c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493599872:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760683.782471:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 19898368, mtime: 0x6aad94ab, atime: 0x0, ctime: 0x6aad94ab, blocks: 0x9807 00010000:00010000:0.0:1789760683.782481:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03b00 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.782718:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=694s v=(0 0) 00010000:00010000:0.0:1789760683.782723:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760683.782728:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03b00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.782743:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03b00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760683.809124:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19898368 -> 19902464 00010000:00010000:0.0:1789760683.809139:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03b00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.809151:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03b00 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.809159:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03b00 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.809166:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760683.809171:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03b00 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.809180:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03b00 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.809198:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03b00 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.809206:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03b00 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760683.809300:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d89c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493599872:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26875us (27227us total) trans 4294972177 rc 0/0 00000100:00100000:0.0:1789760683.809310:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4888 00000100:00100000:0.0:1789760683.809320:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 694074039391 time 694123447155 state 0x2 ref 0 00000100:00100000:0.0:1789760683.813263:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493600000 00000100:00100000:0.0:1789760683.813273:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493600000 00000100:00100000:0.0:1789760683.813327:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 694074039391 time 694123447155 state 0x0 ref 1 00000100:00100000:0.0:1789760683.813342:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 694123707571 time 694123447155 state 0x1 ref 1 00000100:00100000:0.0:1789760683.813374:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 694123707571 time 694154409914 state 0x0 ref 1 00000100:00100000:0.0:1789760683.813387:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493600000:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760683.813442:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 19902464, mtime: 0x6aad94ab, atime: 0x0, ctime: 0x6aad94ab, blocks: 0x9807 00010000:00010000:0.0:1789760683.813450:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e03b07 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.813468:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=694s v=(0 0) 00010000:00010000:0.0:1789760683.813473:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760683.813479:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e03b07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.813493:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e03b07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760683.859380:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19902464 -> 19906560 00010000:00010000:0.0:1789760683.859466:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e03b07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.859479:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e03b07 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.859489:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e03b07 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.859496:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760683.859501:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e03b07 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.859510:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e03b07 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.859530:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e03b07 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760683.859538:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e03b07 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760683.859627:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493600000:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46237us (46690us total) trans 4294972178 rc 0/0 00000100:00100000:0.0:1789760683.859641:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4889 00000100:00100000:0.0:1789760683.859651:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 694123707571 time 694154409914 state 0x2 ref 0 00000100:00100000:0.0:1789760683.862824:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:0.0:1789760683.918948:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972179 is committed 00000001:00080000:0.0:1789760683.956644:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972180 is committed 00000100:00100000:0.0:1789760683.969343:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:0.0:1789760684.058585:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972181 is committed 00000100:00100000:0.0:1789760684.065160:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493600384:12345-192.168.203.60@tcp:4:dd.0 Request processed in 95600us (95820us total) trans 4294972181 rc 0/0 00000100:00100000:0.0:1789760684.065172:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4892 00000100:00100000:0.0:1789760684.065181:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 694265849571 time 694310577612 state 0x2 ref 0 00000100:00100000:0.0:1789760684.069402:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:0.0:1789760684.101841:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972182 is committed 00000100:00100000:0.0:1789760684.108099:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493600640 00000100:00100000:0.0:1789760684.108110:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493600640 00000100:00100000:0.0:1789760684.108166:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 694310838028 time 694410658497 state 0x0 ref 1 00000100:00100000:0.0:1789760684.108181:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 694410918913 time 694410658497 state 0x1 ref 1 00000100:00100000:0.0:1789760684.108215:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 694410918913 time 694449250572 state 0x0 ref 1 00000100:00100000:0.0:1789760684.108228:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493600640:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760684.108277:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 19922944, mtime: 0x6aad94ab, atime: 0x0, ctime: 0x6aad94ab, blocks: 0x9807 00010000:00010000:0.0:1789760684.108286:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03b2a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760684.108304:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=694s v=(0 0) 00010000:00010000:0.0:1789760684.108308:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760684.108314:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03b2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760684.108328:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03b2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760684.158104:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972183 is committed 00000100:00100000:0.0:1789760684.164085:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760684.207590:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760684.243924:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760684.388656:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760684.535365:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493601536 00000100:00100000:0.0:1789760684.535375:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493601536 00000100:00100000:0.0:1789760684.535548:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 694731690303 time 694822099264 state 0x0 ref 1 00000100:00100000:0.0:1789760684.535563:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 694822359680 time 694822099264 state 0x1 ref 1 00000100:00100000:0.0:1789760684.535629:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 694822359680 time 694876664205 state 0x0 ref 1 00000100:00100000:0.0:1789760684.535642:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8556a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493601536:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760684.535689:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 19951616, mtime: 0x6aad94ab, atime: 0x0, ctime: 0x6aad94ab, blocks: 0xa007 00010000:00010000:0.0:1789760684.535698:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e03b5b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760684.535716:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=694s v=(0 0) 00010000:00010000:0.0:1789760684.535721:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760684.535727:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e03b5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760684.535740:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e03b5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760684.583802:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:0.0:1789760684.638858:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972191 is committed 00000100:00100000:0.0:1789760684.639414:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493601664:12345-192.168.203.60@tcp:4:dd.0 Request processed in 55162us (55617us total) trans 4294972191 rc 0/0 00000100:00100000:0.0:1789760684.639426:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4902 00000100:00100000:0.0:1789760684.639436:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 694876924621 time 694925271515 state 0x2 ref 0 00000100:00100000:0.0:1789760684.758406:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493602048 00000100:00100000:0.0:1789760684.758416:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493602048 00000100:00100000:0.0:1789760684.758475:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 694994033690 time 695042044124 state 0x0 ref 1 00000100:00100000:0.0:1789760684.758492:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 695042304540 time 695042044124 state 0x1 ref 1 00000100:00100000:0.0:1789760684.758524:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 695042304540 time 695099559274 state 0x0 ref 1 00000100:00100000:0.0:1789760684.758537:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493602048:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760684.758591:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 19968000, mtime: 0x6aad94ac, atime: 0x0, ctime: 0x6aad94ac, blocks: 0xa007 00010000:00010000:0.0:1789760684.758601:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e03b77 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760684.758617:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=695s v=(0 0) 00010000:00010000:0.0:1789760684.758633:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760684.758640:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e03b77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760684.758655:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e03b77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760684.826119:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 19968000 -> 19972096 00010000:00010000:0.0:1789760684.826142:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e03b77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760684.826158:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e03b77 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760684.826169:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e03b77 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760684.826178:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760684.826184:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e03b77 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760684.826195:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e03b77 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760684.826227:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e03b77 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760684.826236:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e03b77 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760684.826475:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493602048:12345-192.168.203.60@tcp:4:dd.0 Request processed in 67932us (68871us total) trans 4294972194 rc 0/0 00000100:00100000:0.0:1789760684.826489:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4905 00000100:00100000:0.0:1789760684.826501:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 695042304540 time 695099559274 state 0x2 ref 0 00000100:00100000:0.0:1789760684.832474:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:0.0:1789760684.924321:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972195 is committed 00000001:00080000:0.0:1789760685.007847:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972196 is committed 00000100:00100000:0.0:1789760685.014804:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760685.065355:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:0.0:1789760685.117678:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972198 is committed 00000100:00100000:0.0:1789760685.122726:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:0.0:1789760685.196930:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972199 is committed 00000100:00100000:0.0:1789760685.207485:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760685.339063:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:0.0:1789760685.383449:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972201 is committed 00000100:00100000:0.0:1789760685.465657:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760685.948944:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00002000:00010000:0.0:1789760685.982418:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20033536 -> 20037632 00010000:00010000:0.0:1789760685.982432:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e03be7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760685.982444:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e03be7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760685.982453:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e03be7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760685.982460:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760685.982465:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e03be7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760685.982475:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e03be7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760685.982496:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e03be7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760685.982503:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e03be7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760685.982604:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493604096:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33274us (33667us total) trans 4294972210 rc 0/0 00000100:00100000:0.0:1789760685.982614:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4921 00000100:00100000:0.0:1789760685.982625:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 696220954656 time 696290350835 state 0x2 ref 0 00000100:00100000:0.0:1789760686.079416:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760686.118300:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493604608 00000100:00100000:0.0:1789760686.118309:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493604608 00000100:00100000:0.0:1789760686.118360:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 696365473335 time 696420649250 state 0x0 ref 1 00000100:00100000:0.0:1789760686.118375:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 696420909666 time 696420649250 state 0x1 ref 1 00000100:00100000:0.0:1789760686.118419:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 696420909666 time 696459454211 state 0x0 ref 1 00000100:00100000:0.0:1789760686.118432:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493604608:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760686.118478:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20049920, mtime: 0x6aad94ad, atime: 0x0, ctime: 0x6aad94ad, blocks: 0xa007 00010000:00010000:0.0:1789760686.118487:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03c03 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760686.118502:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760686.118506:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760686.118512:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03c03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760686.118525:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03c03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760686.157417:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493604736 00000100:00100000:0.0:1789760686.157426:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493604736 00000100:00100000:0.0:1789760686.157486:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 696420909666 time 696459454211 state 0x0 ref 1 00000100:00100000:0.0:1789760686.157501:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 696459714627 time 696459454211 state 0x1 ref 1 00000100:00100000:0.0:1789760686.157562:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 696459714627 time 696498597410 state 0x0 ref 1 00000100:00100000:0.0:1789760686.157575:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493604736:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760686.157624:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20054016, mtime: 0x6aad94ad, atime: 0x0, ctime: 0x6aad94ad, blocks: 0xa007 00010000:00010000:0.0:1789760686.157633:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03c0a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760686.157648:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760686.157653:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760686.157659:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03c0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760686.157672:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03c0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760686.198999:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972215 is committed 00000001:00080000:0.0:1789760686.254544:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972216 is committed 00002000:00010000:0.0:1789760686.297713:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20062208 -> 20066304 00010000:00010000:0.0:1789760686.297733:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e03c18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760686.297745:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e03c18 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760686.297753:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e03c18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760686.297760:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760686.297765:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e03c18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760686.297774:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e03c18 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760686.297793:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e03c18 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760686.297801:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e03c18 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760686.297888:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9621016d0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493605376:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38886us (39685us total) trans 4294972217 rc 0/0 00000100:00100000:0.0:1789760686.297906:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4928 00000100:00100000:0.0:1789760686.297916:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 696547422152 time 696599979112 state 0x2 ref 0 00000001:00080000:0.0:1789760686.298222:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972217 is committed 00000100:00100000:0.0:1789760686.304988:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:0.0:1789760686.355512:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972218 is committed 00000001:00080000:0.0:1789760686.432287:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972219 is committed 00000100:00100000:0.0:1789760686.437764:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760686.528508:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493605888 00000100:00100000:0.0:1789760686.528519:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493605888 00000100:00100000:0.0:1789760686.528636:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 696703314298 time 696779196707 state 0x0 ref 1 00000100:00100000:0.0:1789760686.528653:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 696779457123 time 696779196707 state 0x1 ref 1 00000100:00100000:0.0:1789760686.528689:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 696779457123 time 696869724770 state 0x0 ref 1 00000100:00100000:0.0:1789760686.528702:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493605888:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760686.528751:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20078592, mtime: 0x6aad94ad, atime: 0x0, ctime: 0x6aad94ad, blocks: 0xa007 00010000:00010000:0.0:1789760686.528760:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03c34 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760686.528776:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760686.528781:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760686.528799:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03c34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760686.528812:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03c34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760686.581458:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972221 is committed 00000100:00100000:0.0:1789760686.589304:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493606016 00000100:00100000:0.0:1789760686.589313:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493606016 00000100:00100000:0.0:1789760686.589379:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 696779457123 time 696869724770 state 0x0 ref 1 00000100:00100000:0.0:1789760686.589395:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 696869985186 time 696869724770 state 0x1 ref 1 00000100:00100000:0.0:1789760686.589424:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 696869985186 time 696930459355 state 0x0 ref 1 00000100:00100000:0.0:1789760686.589437:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493606016:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760686.589488:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20082688, mtime: 0x6aad94ad, atime: 0x0, ctime: 0x6aad94ad, blocks: 0xa007 00010000:00010000:0.0:1789760686.589497:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03c3b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760686.589513:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760686.589517:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760686.589523:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03c3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760686.589540:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03c3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760686.663110:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760686.908811:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493606528 00000100:00100000:0.0:1789760686.908821:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493606528 00000100:00100000:0.0:1789760686.908879:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 697063423384 time 697179274291 state 0x0 ref 1 00000100:00100000:0.0:1789760686.908906:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 697179534707 time 697179274291 state 0x1 ref 1 00000100:00100000:0.0:1789760686.909031:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 697179534707 time 697250064796 state 0x0 ref 1 00000100:00100000:0.0:1789760686.909045:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493606528:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760686.909098:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20099072, mtime: 0x6aad94ae, atime: 0x0, ctime: 0x6aad94ae, blocks: 0xa007 00010000:00010000:0.0:1789760686.909107:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03c57 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760686.909125:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760686.909129:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760686.909134:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03c57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760686.909144:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03c57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760686.958859:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 697179534707 time 697250064796 state 0x0 ref 1 00000100:00100000:0.0:1789760686.958998:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 697250325212 time 697250064796 state 0x1 ref 1 00000100:00100000:0.0:1789760686.959110:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 697250325212 time 697300144367 state 0x0 ref 1 00000100:00100000:0.0:1789760686.959127:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493606656:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760686.959183:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20103168, mtime: 0x6aad94ae, atime: 0x0, ctime: 0x6aad94ae, blocks: 0xa007 00010000:00010000:0.0:1789760686.959192:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e03c5e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760686.959208:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760686.959212:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760686.959218:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e03c5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760686.959231:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e03c5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760687.028717:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972227 is committed 00000100:00100000:0.0:1789760687.159384:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493607040 00000100:00100000:0.0:1789760687.159394:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493607040 00000100:00100000:0.0:1789760687.159450:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 697390001139 time 697447210865 state 0x0 ref 1 00000100:00100000:0.0:1789760687.159466:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 697447471281 time 697447210865 state 0x1 ref 1 00000100:00100000:0.0:1789760687.159500:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 697447471281 time 697500534827 state 0x0 ref 1 00000100:00100000:0.0:1789760687.159513:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8794700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493607040:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760687.159561:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20115456, mtime: 0x6aad94ae, atime: 0x0, ctime: 0x6aad94ae, blocks: 0xa007 00010000:00010000:0.0:1789760687.159569:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03c73 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.159584:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760687.159587:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760687.159593:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03c73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.159606:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03c73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760687.237211:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972230 is committed 00000100:00100000:0.0:1789760687.246039:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493607168 00000100:00100000:0.0:1789760687.246049:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493607168 00000100:00100000:0.0:1789760687.246105:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 697447471281 time 697500534827 state 0x0 ref 1 00000100:00100000:0.0:1789760687.246122:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 697500795243 time 697500534827 state 0x1 ref 1 00000100:00100000:0.0:1789760687.246157:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 697500795243 time 697587191985 state 0x0 ref 1 00000100:00100000:0.0:1789760687.246170:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493607168:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760687.246223:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20119552, mtime: 0x6aad94ae, atime: 0x0, ctime: 0x6aad94ae, blocks: 0xa007 00010000:00010000:0.0:1789760687.246232:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e03c7a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.246248:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760687.246252:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760687.246258:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e03c7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.246271:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e03c7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760687.392156:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493607424 00000100:00100000:0.0:1789760687.392166:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493607424 00000100:00100000:0.0:1789760687.392219:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 697587452401 time 697661446459 state 0x0 ref 1 00000100:00100000:0.0:1789760687.392236:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 697661706875 time 697661446459 state 0x1 ref 1 00000100:00100000:0.0:1789760687.392271:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 697661706875 time 697733306754 state 0x0 ref 1 00000100:00100000:0.0:1789760687.392284:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493607424:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760687.392333:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20127744, mtime: 0x6aad94ae, atime: 0x0, ctime: 0x6aad94ae, blocks: 0xa007 00010000:00010000:0.0:1789760687.392342:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03c88 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.392357:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760687.392371:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760687.392377:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03c88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.392391:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03c88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760687.423562:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493607552 00000100:00100000:0.0:1789760687.423572:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493607552 00000100:00100000:0.0:1789760687.423628:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 697661706875 time 697733306754 state 0x0 ref 1 00000100:00100000:0.0:1789760687.423643:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 697733567170 time 697733306754 state 0x1 ref 1 00000100:00100000:0.0:1789760687.423673:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 697733567170 time 697764707697 state 0x0 ref 1 00000100:00100000:0.0:1789760687.423686:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8796a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493607552:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760687.423732:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20131840, mtime: 0x6aad94ae, atime: 0x0, ctime: 0x6aad94ae, blocks: 0xa007 00010000:00010000:0.0:1789760687.423741:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e03c8f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.423756:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760687.423761:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760687.423768:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e03c8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.423781:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e03c8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760687.486175:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620c8557b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3305:1876699519242880:0@lo:400:kworker.0 00000100:00100000:0.0:1789760687.486207:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760687.486281:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1789760687.486288:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620c8556a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3305:1876699519242496:0@lo:400:kworker.0 00000100:00100000:0.0:1789760687.486318:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760687.486338:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620c8555180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519243008:0@lo:400:kworker.0 00000100:00100000:0.0:1789760687.486373:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000001:00080000:0.0:1789760687.561724:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972236 is committed 00002000:00010000:0.0:1789760687.562321:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20140032 -> 20144128 00010000:00010000:0.0:1789760687.562338:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e03c9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.562352:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e03c9d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.562360:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e03c9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.562368:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760687.562373:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e03c9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.562383:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e03c9d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.562402:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e03c9d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.562411:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e03c9d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760687.562542:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493607808:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38321us (38576us total) trans 4294972236 rc 0/0 00000100:00100000:0.0:1789760687.562553:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4947 00000100:00100000:0.0:1789760687.562563:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 697823356467 time 697865241804 state 0x2 ref 0 00002000:00010000:0.0:1789760687.575241:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20144128, mtime: 0x6aad94ae, atime: 0x0, ctime: 0x6aad94ae, blocks: 0xa007 00010000:00010000:0.0:1789760687.575251:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03ca4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.575266:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760687.575269:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760687.575275:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03ca4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.575289:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03ca4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760687.673127:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493608064 00000100:00100000:0.0:1789760687.673137:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493608064 00000100:00100000:0.0:1789760687.673201:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 697865502220 time 697915945166 state 0x0 ref 1 00000100:00100000:0.0:1789760687.673216:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 697916205582 time 697915945166 state 0x1 ref 1 00000100:00100000:0.0:1789760687.673248:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 697916205582 time 698014283308 state 0x0 ref 1 00000100:00100000:0.0:1789760687.673260:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c89f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493608064:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760687.673307:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20148224, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0xa007 00010000:00010000:0.0:1789760687.673316:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e03cab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.673331:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760687.673346:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760687.673353:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e03cab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.673368:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e03cab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760687.738382:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20148224 -> 20152320 00010000:00010000:0.0:1789760687.738397:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e03cab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.738418:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e03cab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.738427:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e03cab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.738436:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760687.738440:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e03cab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.738450:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e03cab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.738470:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e03cab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.738478:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e03cab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760687.738611:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c89f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493608064:12345-192.168.203.60@tcp:4:dd.0 Request processed in 65348us (65573us total) trans 4294972238 rc 0/0 00000100:00100000:0.0:1789760687.738622:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4949 00000100:00100000:0.0:1789760687.738631:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 697916205582 time 698014283308 state 0x2 ref 0 00000100:00100000:0.0:1789760687.790643:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493608320 00000100:00100000:0.0:1789760687.790654:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493608320 00000100:00100000:0.0:1789760687.790710:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 698014543724 time 698085550670 state 0x0 ref 1 00000100:00100000:0.0:1789760687.790726:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 698085811086 time 698085550670 state 0x1 ref 1 00000100:00100000:0.0:1789760687.790809:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 698085811086 time 698131843516 state 0x0 ref 1 00000100:00100000:0.0:1789760687.790822:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493608320:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760687.790870:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20156416, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0xa007 00010000:00010000:0.0:1789760687.790890:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e03cb9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.790906:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760687.790911:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760687.790917:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e03cb9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.790940:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e03cb9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760687.857035:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493608448 00000100:00100000:0.0:1789760687.857045:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493608448 00000100:00100000:0.0:1789760687.857100:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 698085811086 time 698131843516 state 0x0 ref 1 00000100:00100000:0.0:1789760687.857114:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 698132103932 time 698131843516 state 0x1 ref 1 00000100:00100000:0.0:1789760687.857146:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 698132103932 time 698198181905 state 0x0 ref 1 00000100:00100000:0.0:1789760687.857163:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620f39f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493608448:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760687.857228:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20160512, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0xa007 00010000:00010000:0.0:1789760687.857237:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03cc0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.857252:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760687.857256:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760687.857262:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03cc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.857274:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03cc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760687.939757:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493608576 00000100:00100000:0.0:1789760687.939772:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493608576 00000100:00100000:0.0:1789760687.939839:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 698132103932 time 698198181905 state 0x0 ref 1 00000100:00100000:0.0:1789760687.939854:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 698198442321 time 698198181905 state 0x1 ref 1 00000100:00100000:0.0:1789760687.939899:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 698198442321 time 698280935278 state 0x0 ref 1 00000100:00100000:0.0:1789760687.939912:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493608576:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760687.940000:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20164608, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0xa007 00010000:00010000:0.0:1789760687.940011:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e03cc7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.940033:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760687.940038:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760687.940045:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e03cc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760687.940059:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e03cc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760688.004149:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493608704 00000100:00100000:0.0:1789760688.004162:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493608704 00000100:00100000:0.0:1789760688.004220:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 698198442321 time 698280935278 state 0x0 ref 1 00000100:00100000:0.0:1789760688.004236:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 698281195694 time 698280935278 state 0x1 ref 1 00000100:00100000:0.0:1789760688.004301:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 698281195694 time 698345335462 state 0x0 ref 1 00000100:00100000:0.0:1789760688.004313:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8557b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493608704:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760688.004378:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20168704, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0xa007 00010000:00010000:0.0:1789760688.004387:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e03cce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.004401:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760688.004406:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760688.004413:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e03cce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.004431:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e03cce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760688.242313:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00002000:00010000:0.0:1789760688.287386:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20185088 -> 20189184 00010000:00010000:0.0:1789760688.287402:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03cea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.287481:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03cea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.287491:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03cea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.287500:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760688.287505:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03cea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.287515:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03cea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.287536:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03cea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.287545:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03cea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760688.287669:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493609216:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45153us (45360us total) trans 4294972247 rc 0/0 00000100:00100000:0.0:1789760688.287680:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4958 00000100:00100000:0.0:1789760688.287690:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 698544697240 time 698583535974 state 0x2 ref 0 00000100:00100000:0.0:1789760688.291133:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:0.0:1789760688.329849:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972248 is committed 00000001:00080000:0.0:1789760688.391704:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972249 is committed 00000100:00100000:0.0:1789760688.396352:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:0.0:1789760688.435589:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972250 is committed 00000100:00100000:0.0:1789760688.446576:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:0.0:1789760688.489445:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972251 is committed 00000100:00100000:0.0:1789760688.494201:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493609856 00000100:00100000:0.0:1789760688.494211:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493609856 00000100:00100000:0.0:1789760688.494266:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 698737923206 time 698787835429 state 0x0 ref 1 00000100:00100000:0.0:1789760688.494281:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 698788095845 time 698787835429 state 0x1 ref 1 00000100:00100000:0.0:1789760688.494314:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 698788095845 time 698835348990 state 0x0 ref 1 00000100:00100000:0.0:1789760688.494327:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8556300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493609856:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760688.494377:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20205568, mtime: 0x6aad94af, atime: 0x0, ctime: 0x6aad94af, blocks: 0xa007 00010000:00010000:0.0:1789760688.494386:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e03d0d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.494418:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760688.494423:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760688.494430:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e03d0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.494443:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e03d0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760688.532904:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972252 is committed 00000100:00100000:0.0:1789760688.585132:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760688.703372:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493610368 00000100:00100000:0.0:1789760688.703382:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493610368 00000100:00100000:0.0:1789760688.703577:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 698926630094 time 698986585533 state 0x0 ref 1 00000100:00100000:0.0:1789760688.703592:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 698986845949 time 698986585533 state 0x1 ref 1 00000100:00100000:0.0:1789760688.703625:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 698986845949 time 699044660925 state 0x0 ref 1 00000100:00100000:0.0:1789760688.703638:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493610368:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760688.703688:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20221952, mtime: 0x6aad94b0, atime: 0x0, ctime: 0x6aad94b0, blocks: 0xa007 00010000:00010000:0.0:1789760688.703697:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e03d29 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.703715:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760688.703720:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760688.703727:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e03d29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.703746:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e03d29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760688.767885:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20221952 -> 20226048 00010000:00010000:0.0:1789760688.767899:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e03d29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.767912:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e03d29 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.767921:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e03d29 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.767937:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760688.767942:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e03d29 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.767951:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e03d29 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.767987:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e03d29 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760688.767996:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e03d29 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760688.768151:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493610368:12345-192.168.203.60@tcp:4:dd.0 Request processed in 64509us (65401us total) trans 4294972256 rc 0/0 00000100:00100000:0.0:1789760688.768162:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4967 00000100:00100000:0.0:1789760688.768171:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 698986845949 time 699044660925 state 0x2 ref 0 00000100:00100000:0.0:1789760688.773462:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760689.173798:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:0.0:1789760689.290519:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972262 is committed 00000100:00100000:0.0:1789760689.294625:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:0.0:1789760689.415324:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972265 is committed 00000100:00100000:0.0:1789760689.429401:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00002000:00010000:0.0:1789760689.494445:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20262912 -> 20267008 00010000:00010000:0.0:1789760689.494468:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03d6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.494482:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03d6f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.494491:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03d6f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.494499:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760689.494504:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03d6f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.494531:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03d6f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.494555:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03d6f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.494563:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03d6f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760689.494679:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493611648:12345-192.168.203.60@tcp:4:dd.0 Request processed in 65033us (65283us total) trans 4294972266 rc 0/0 00000100:00100000:0.0:1789760689.494690:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4977 00000100:00100000:0.0:1789760689.494702:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 699716023921 time 699770664932 state 0x2 ref 0 00000001:00080000:0.0:1789760689.494755:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972266 is committed 00000001:00080000:0.0:1789760689.590565:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972267 is committed 00000001:00080000:0.0:1789760689.649331:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972268 is committed 00000100:00100000:0.0:1789760689.652771:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493612032 00000100:00100000:0.0:1789760689.652781:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493612032 00000100:00100000:0.0:1789760689.652842:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 699845174603 time 699937382943 state 0x0 ref 1 00000100:00100000:0.0:1789760689.652856:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 699937643359 time 699937382943 state 0x1 ref 1 00000100:00100000:0.0:1789760689.652893:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 699937643359 time 699993928391 state 0x0 ref 1 00000100:00100000:0.0:1789760689.652904:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493612032:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760689.653043:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20275200, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0xa007 00010000:00010000:0.0:1789760689.653052:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e03d84 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.653068:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760689.653073:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760689.653078:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e03d84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.653092:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e03d84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760689.674636:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972269 is committed 00000100:00100000:0.0:1789760689.682694:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493612160 00000100:00100000:0.0:1789760689.682705:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493612160 00000100:00100000:0.0:1789760689.682763:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 699937643359 time 699993928391 state 0x0 ref 1 00000100:00100000:0.0:1789760689.682779:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 699994188807 time 699993928391 state 0x1 ref 1 00000100:00100000:0.0:1789760689.682809:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 699994188807 time 700023845171 state 0x0 ref 1 00000100:00100000:0.0:1789760689.682820:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c3097b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493612160:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760689.682879:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20279296, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0xa007 00010000:00010000:0.0:1789760689.682888:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03d8b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.682909:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760689.682913:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760689.682918:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03d8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.682932:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03d8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760689.719650:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20279296 -> 20283392 00010000:00010000:0.0:1789760689.719664:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03d8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.719678:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03d8b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.719686:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03d8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.719693:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760689.719698:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03d8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.719707:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03d8b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.719726:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03d8b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.719734:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03d8b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760689.719851:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c3097b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493612160:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37028us (37385us total) trans 4294972270 rc 0/0 00000100:00100000:0.0:1789760689.719863:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4981 00000100:00100000:0.0:1789760689.719882:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 699994188807 time 700023845171 state 0x2 ref 0 00000100:00100000:0.0:1789760689.723799:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00002000:00010000:0.0:1789760689.757233:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20283392 -> 20287488 00010000:00010000:0.0:1789760689.757248:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e03d92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.757261:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e03d92 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.757270:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e03d92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.757277:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760689.757281:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e03d92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.757289:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e03d92 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.757313:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e03d92 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.757321:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e03d92 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760689.757640:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493612288:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33393us (33836us total) trans 4294972271 rc 0/0 00000100:00100000:0.0:1789760689.757651:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4982 00000100:00100000:0.0:1789760689.757660:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 700024105587 time 700065257463 state 0x2 ref 0 00002000:00010000:0.0:1789760689.796358:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20287488 -> 20291584 00010000:00010000:0.0:1789760689.796378:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03d99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.796391:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03d99 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.796400:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03d99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.796409:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760689.796414:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03d99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.796428:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03d99 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.796448:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03d99 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760689.796456:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03d99 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760689.796545:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c3097800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493612416:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33970us (34433us total) trans 4294972272 rc 0/0 00000100:00100000:0.0:1789760689.796556:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4983 00000100:00100000:0.0:1789760689.796566:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 700065517879 time 700103592237 state 0x2 ref 0 00000001:00080000:0.0:1789760689.835052:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972273 is committed 00000100:00100000:0.0:1789760689.842697:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760689.960393:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760690.019218:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493612928 00000100:00100000:0.0:1789760690.019229:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493612928 00000100:00100000:0.0:1789760690.019284:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 700184249998 time 700301689556 state 0x0 ref 1 00000100:00100000:0.0:1789760690.019300:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 700301949972 time 700301689556 state 0x1 ref 1 00000100:00100000:0.0:1789760690.019517:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 700301949972 time 700360550791 state 0x0 ref 1 00000100:00100000:0.0:1789760690.019534:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493612928:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760690.019584:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20303872, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0xa007 00010000:00010000:0.0:1789760690.019592:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e03db5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.019609:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760690.019613:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760690.019620:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e03db5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.019646:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e03db5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760690.070135:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760690.110777:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00002000:00010000:0.0:1789760690.135838:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20312064 -> 20316160 00010000:00010000:0.0:1789760690.135853:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e03dc3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.135876:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e03dc3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.135885:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e03dc3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.135894:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760690.135899:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e03dc3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.135909:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e03dc3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.135937:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e03dc3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.135945:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e03dc3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760690.136098:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493613184:12345-192.168.203.60@tcp:4:dd.0 Request processed in 24877us (25323us total) trans 4294972278 rc 0/0 00000100:00100000:0.0:1789760690.136109:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4989 00000100:00100000:0.0:1789760690.136118:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 700416879670 time 700452236677 state 0x2 ref 0 00000100:00100000:0.0:1789760690.143840:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:0.0:1789760690.200512:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972279 is committed 00002000:00010000:0.0:1789760690.247244:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20320256 -> 20324352 00010000:00010000:0.0:1789760690.247263:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03dd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.247276:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03dd1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.247284:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03dd1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.247296:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760690.247301:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03dd1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.247310:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03dd1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.247328:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03dd1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.247335:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03dd1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760690.247442:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493613440:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41941us (42513us total) trans 4294972280 rc 0/0 00000100:00100000:0.0:1789760690.247453:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4991 00000100:00100000:0.0:1789760690.247464:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 700485394646 time 700546518906 state 0x2 ref 0 00000100:00100000:0.0:1789760690.253075:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493613568 00000100:00100000:0.0:1789760690.253086:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493613568 00000100:00100000:0.0:1789760690.253143:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 700485394646 time 700546518906 state 0x0 ref 1 00000100:00100000:0.0:1789760690.253158:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 700546779322 time 700546518906 state 0x1 ref 1 00000100:00100000:0.0:1789760690.253229:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 700546779322 time 700594263508 state 0x0 ref 1 00000100:00100000:0.0:1789760690.253245:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493613568:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760690.253294:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20324352, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0xa007 00010000:00010000:0.0:1789760690.253303:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03dd8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.253317:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760690.253322:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760690.253328:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03dd8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.253351:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03dd8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760690.305134:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493613696 00000100:00100000:0.0:1789760690.305143:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493613696 00000100:00100000:0.0:1789760690.305195:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 700546779322 time 700594263508 state 0x0 ref 1 00000100:00100000:0.0:1789760690.305210:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 700594523924 time 700594263508 state 0x1 ref 1 00000100:00100000:0.0:1789760690.305238:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 700594523924 time 700646274518 state 0x0 ref 1 00000100:00100000:0.0:1789760690.305250:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c3097480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493613696:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760690.305296:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20328448, mtime: 0x6aad94b1, atime: 0x0, ctime: 0x6aad94b1, blocks: 0xa007 00010000:00010000:0.0:1789760690.305305:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e03ddf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.305326:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760690.305330:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760690.305336:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e03ddf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.305359:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e03ddf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.375190:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03de6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760690.438685:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20332544 -> 20336640 00010000:00010000:2.0:1789760690.438708:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03de6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.438722:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03de6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.438731:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03de6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.438739:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760690.438744:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03de6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.438753:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03de6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.438771:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03de6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.438778:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03de6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760690.438883:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493613824:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63785us (64063us total) trans 4294972283 rc 0/0 00000100:00100000:2.0:1789760690.438936:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4994 00000100:00100000:2.0:1789760690.438955:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 700646534934 time 700716119049 state 0x2 ref 0 00000001:00080000:2.0:1789760690.488725:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972284 is committed 00000001:00080000:0.0:1789760690.549045:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972285 is committed 00000001:00080000:0.0:1789760690.602455:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972286 is committed 00002000:00010000:2.0:1789760690.603181:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20344832 -> 20348928 00010000:00010000:2.0:1789760690.603198:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03dfb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.603212:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03dfb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.603222:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03dfb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.603231:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760690.603235:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03dfb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.603244:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03dfb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.603265:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03dfb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.603273:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03dfb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760690.603539:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493614208:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49268us (50104us total) trans 4294972286 rc 0/0 00000100:00100000:2.0:1789760690.603551:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4997 00000100:00100000:2.0:1789760690.603561:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 700834776387 time 700895288803 state 0x2 ref 0 00000100:00100000:0.0:1789760690.607582:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:2.0:1789760690.667748:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972287 is committed 00000100:00100000:2.0:1789760690.672490:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760690.673714:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 700895549219 time 700948843391 state 0x0 ref 1 00000100:00100000:0.0:1789760690.673730:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 700949103807 time 700948843391 state 0x1 ref 1 00000100:00100000:0.0:1789760690.673791:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 700949103807 time 701014826880 state 0x0 ref 1 00000100:00100000:0.0:1789760690.673812:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c3095880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493614464:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760690.673871:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20353024, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0xa007 00010000:00010000:0.0:1789760690.673879:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03e09 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.673894:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760690.673898:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760690.673904:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03e09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.673928:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03e09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.682219:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760690.733248:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20353024 -> 20357120 00010000:00010000:2.0:1789760690.733264:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03e09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.733277:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03e09 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.733286:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03e09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.733293:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760690.733298:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03e09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.733307:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03e09 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.733325:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03e09 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.733334:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e03e09 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760690.733502:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c3095880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493614464:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59688us (61018us total) trans 4294972288 rc 0/0 00000100:00100000:2.0:1789760690.733584:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 4999 00000100:00100000:2.0:1789760690.733596:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 700949103807 time 701014826880 state 0x2 ref 0 00000001:00080000:2.0:1789760690.785577:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972289 is committed 00000100:00100000:2.0:1789760690.791330:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493614720 00000100:00100000:2.0:1789760690.791408:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493614720 00000100:00100000:2.0:1789760690.791466:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701015087296 time 701079626392 state 0x0 ref 1 00000100:00100000:2.0:1789760690.791481:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701079886808 time 701079626392 state 0x1 ref 1 00000100:00100000:2.0:1789760690.791610:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701079886808 time 701132644347 state 0x0 ref 1 00000100:00100000:2.0:1789760690.791626:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb463b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493614720:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760690.791677:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20361216, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0xa007 00010000:00010000:2.0:1789760690.791685:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e03e17 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.791701:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:2.0:1789760690.791706:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760690.791712:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e03e17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.791728:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e03e17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760690.851231:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972290 is committed 00000100:00100000:2.0:1789760690.859671:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760690.860504:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701132904763 time 701201535468 state 0x0 ref 1 00000100:00100000:2.0:1789760690.860609:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c3097b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493614848:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760690.860670:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20365312, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0xa007 00010000:00010000:2.0:1789760690.860679:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e03e1e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.860695:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:2.0:1789760690.860699:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760690.860705:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e03e1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760690.860720:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e03e1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760690.929168:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760690.929565:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493614976 00000100:00100000:0.0:1789760690.929575:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493614976 00000100:00100000:0.0:1789760690.929631:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701132904763 time 701201535468 state 0x0 ref 1 00000100:00100000:0.0:1789760690.929651:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701201795884 time 701201535468 state 0x1 ref 1 00000100:00100000:0.0:1789760690.929687:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701201795884 time 701270723423 state 0x0 ref 1 00000100:00100000:0.0:1789760690.929700:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c3097800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493614976:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760690.929748:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20369408, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0xa007 00010000:00010000:0.0:1789760690.929757:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03e25 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.929772:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760690.929776:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760690.929782:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03e25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760690.929800:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03e25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760691.019665:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00010000:00010000:1.0:1789760691.020062:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e03e2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.020082:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e03e2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760691.021200:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789760691.100875:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20373504 -> 20377600 00010000:00010000:0.0:1789760691.100894:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e03e2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.100916:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e03e2c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.100925:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e03e2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.100935:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760691.100940:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e03e2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.100949:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e03e2c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.100994:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e03e2c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.101002:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e03e2c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760691.101208:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493615104:12345-192.168.203.60@tcp:4:dd.0 Request processed in 81278us (81547us total) trans 4294972293 rc 0/0 00000100:00100000:0.0:1789760691.101222:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5004 00000100:00100000:0.0:1789760691.101236:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701270983839 time 701360939752 state 0x2 ref 0 00000100:00100000:0.0:1789760691.104750:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760691.104818:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493615232 00000100:00100000:1.0:1789760691.104828:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493615232 00000100:00100000:1.0:1789760691.104898:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701270983839 time 701360939752 state 0x0 ref 1 00000100:00100000:1.0:1789760691.104920:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701361200168 time 701360939752 state 0x1 ref 1 00000100:00100000:1.0:1789760691.104945:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701361200168 time 701445981363 state 0x0 ref 1 00000100:00100000:1.0:1789760691.104956:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493615232:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760691.105023:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20377600, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0xa007 00010000:00010000:1.0:1789760691.105032:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03e33 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.105047:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760691.105051:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760691.105057:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03e33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.105070:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03e33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760691.148481:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20377600 -> 20381696 00010000:00010000:1.0:1789760691.148501:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03e33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.148514:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03e33 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.148523:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03e33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.148529:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760691.148534:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03e33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.148543:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03e33 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.148562:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03e33 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.148570:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03e33 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760691.148650:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493615232:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43691us (43905us total) trans 4294972294 rc 0/0 00000100:00100000:1.0:1789760691.148660:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5005 00000100:00100000:1.0:1789760691.148669:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701361200168 time 701445981363 state 0x2 ref 0 00000100:00100000:1.0:1789760691.152578:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00002000:00010000:0.0:1789760691.202347:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20381696 -> 20385792 00010000:00010000:0.0:1789760691.202368:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e03e3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.202381:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e03e3a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.202391:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e03e3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.202398:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760691.202403:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e03e3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.202413:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e03e3a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.202433:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e03e3a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.202442:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e03e3a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760691.202548:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493615360:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49658us (49973us total) trans 4294972295 rc 0/0 00000100:00100000:0.0:1789760691.202559:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5006 00000100:00100000:0.0:1789760691.202569:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701446241779 time 701493909955 state 0x2 ref 0 00000100:00100000:1.0:1789760691.210638:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00002000:00010000:0.0:1789760691.267573:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20385792 -> 20389888 00010000:00010000:0.0:1789760691.267594:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03e41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.267608:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03e41 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.267616:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03e41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.267624:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760691.267628:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03e41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.267637:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03e41 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.267657:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03e41 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.267670:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03e41 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760691.267762:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493615488:12345-192.168.203.60@tcp:4:dd.0 Request processed in 56941us (57125us total) trans 4294972296 rc 0/0 00000100:00100000:0.0:1789760691.267772:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5007 00000100:00100000:0.0:1789760691.267782:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701494170371 time 701551840444 state 0x2 ref 0 00000100:00100000:1.0:1789760691.271509:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760691.271588:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493615616 00000100:00100000:0.0:1789760691.271597:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493615616 00000100:00100000:0.0:1789760691.271648:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701494170371 time 701551840444 state 0x0 ref 1 00000100:00100000:0.0:1789760691.271661:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701552100860 time 701551840444 state 0x1 ref 1 00000100:00100000:0.0:1789760691.271695:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701552100860 time 701612730087 state 0x0 ref 1 00000100:00100000:0.0:1789760691.271708:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493615616:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760691.271755:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20389888, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0xa007 00010000:00010000:0.0:1789760691.271763:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03e48 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.271779:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760691.271783:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760691.271789:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03e48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.271808:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03e48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760691.307652:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760691.307686:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760691.307724:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760691.307753:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493616000 00000100:00100000:2.0:1789760691.307763:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493616000 00000100:00100000:2.0:1789760691.307831:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493615872 00000100:00100000:2.0:1789760691.307834:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493615872 00000100:00100000:2.0:1789760691.307871:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 432 00000100:00100000:2.0:1789760691.307881:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c3097100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2367:x1876699493616000:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:2.0:1789760691.307994:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c3097100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2367:x1876699493616000:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 114us (307us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760691.308003:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 432 00000100:00100000:2.0:1789760691.308020:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 433 00000100:00100000:2.0:1789760691.308027:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a34380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2367:x1876699493615872:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:2.0:1789760691.308051:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a34380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2367:x1876699493615872:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 25us (326us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760691.308057:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 433 00000100:00080000:2.0:1789760691.308075:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00002000:00010000:1.0:1789760691.314250:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20389888 -> 20393984 00010000:00010000:1.0:1789760691.314267:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03e48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.314279:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03e48 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.314289:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03e48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.314296:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760691.314301:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03e48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.314310:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03e48 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.314329:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03e48 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.314348:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e03e48 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760691.314443:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493615616:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42733us (42938us total) trans 4294972297 rc 0/0 00000100:00100000:1.0:1789760691.314454:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5008 00000100:00100000:1.0:1789760691.314464:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701552100860 time 701612730087 state 0x2 ref 0 00000100:00100000:2.0:1789760691.317906:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760691.318030:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493616128 00000100:00100000:1.0:1789760691.318041:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493616128 00000100:00100000:1.0:1789760691.318101:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701552100860 time 701612730087 state 0x0 ref 1 00000100:00100000:1.0:1789760691.318116:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701612990503 time 701612730087 state 0x1 ref 1 00000100:00100000:1.0:1789760691.318135:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701612990503 time 701659171641 state 0x0 ref 1 00000100:00100000:1.0:1789760691.318147:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493616128:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760691.318195:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20393984, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0xa007 00010000:00010000:1.0:1789760691.318205:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e03e4f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.318221:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760691.318225:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760691.318231:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e03e4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.318247:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e03e4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760691.346729:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20393984 -> 20398080 00010000:00010000:1.0:1789760691.346745:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e03e4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.346758:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e03e4f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.346766:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e03e4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.346773:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760691.346778:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e03e4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.346786:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e03e4f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.346806:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e03e4f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.346813:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e03e4f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760691.347015:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493616128:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28761us (29011us total) trans 4294972298 rc 0/0 00000100:00100000:1.0:1789760691.347025:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5009 00000100:00100000:1.0:1789760691.347035:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701612990503 time 701659171641 state 0x2 ref 0 00000100:00100000:2.0:1789760691.351019:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:2.0:1789760691.386246:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972299 is committed 00000100:00100000:0.0:1789760691.388620:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493616384 00000100:00100000:0.0:1789760691.388631:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493616384 00000100:00100000:0.0:1789760691.388684:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701659432057 time 701692248416 state 0x0 ref 1 00000100:00100000:0.0:1789760691.388698:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701692508832 time 701692248416 state 0x1 ref 1 00000100:00100000:0.0:1789760691.388731:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701692508832 time 701729766639 state 0x0 ref 1 00000100:00100000:0.0:1789760691.388743:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493616384:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760691.388791:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20402176, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0xa007 00010000:00010000:0.0:1789760691.388801:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e03e5d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.388816:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760691.388820:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760691.388826:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e03e5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.388841:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e03e5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760691.428358:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972300 is committed 00000100:00100000:1.0:1789760691.431482:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760691.431559:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493616512 00000100:00100000:0.0:1789760691.431571:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493616512 00000100:00100000:0.0:1789760691.431634:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701692508832 time 701729766639 state 0x0 ref 1 00000100:00100000:0.0:1789760691.431650:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701730027055 time 701729766639 state 0x1 ref 1 00000100:00100000:0.0:1789760691.431681:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701730027055 time 701772717169 state 0x0 ref 1 00000100:00100000:0.0:1789760691.431693:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493616512:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760691.431743:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20406272, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0xa007 00010000:00010000:0.0:1789760691.431750:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03e64 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.431766:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760691.431771:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760691.431777:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03e64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.431790:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03e64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760691.455368:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20406272 -> 20410368 00010000:00010000:1.0:1789760691.455387:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03e64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.455400:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03e64 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.455410:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03e64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.455417:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760691.455422:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03e64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.455431:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03e64 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.455450:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03e64 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.455457:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e03e64 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760691.455548:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493616512:12345-192.168.203.60@tcp:4:dd.0 Request processed in 23852us (24072us total) trans 4294972301 rc 0/0 00000100:00100000:1.0:1789760691.455558:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5012 00000100:00100000:1.0:1789760691.455567:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701730027055 time 701772717169 state 0x2 ref 0 00000001:00080000:2.0:1789760691.457189:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972301 is committed 00000100:00100000:2.0:1789760691.459932:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:2.0:1789760691.508157:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972302 is committed 00002000:00010000:0.0:1789760691.508688:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20410368 -> 20414464 00010000:00010000:0.0:1789760691.508704:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03e6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.508717:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03e6b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.508732:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03e6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.508739:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760691.508744:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03e6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.508758:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03e6b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.508778:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03e6b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.508786:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03e6b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760691.508895:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a34e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493616640:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48416us (48971us total) trans 4294972302 rc 0/0 00000100:00100000:0.0:1789760691.511838:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5013 00000100:00100000:0.0:1789760691.511863:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701772977585 time 701801494243 state 0x2 ref 0 00000100:00100000:1.0:1789760691.514442:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:0.0:1789760691.550139:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972303 is committed 00000100:00100000:1.0:1789760691.553606:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760691.553689:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493616896 00000100:00100000:0.0:1789760691.553700:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493616896 00000100:00100000:0.0:1789760691.553754:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701801754659 time 701855876254 state 0x0 ref 1 00000100:00100000:0.0:1789760691.553772:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701856136670 time 701855876254 state 0x1 ref 1 00000100:00100000:0.0:1789760691.553805:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701856136670 time 701894840850 state 0x0 ref 1 00000100:00100000:0.0:1789760691.553819:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493616896:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760691.553880:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20418560, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0xa007 00010000:00010000:0.0:1789760691.553888:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e03e79 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.553903:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760691.553918:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760691.553924:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e03e79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.553936:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e03e79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760691.579555:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20418560 -> 20422656 00010000:00010000:0.0:1789760691.579573:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e03e79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.579584:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e03e79 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.579592:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e03e79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.579604:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760691.579609:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e03e79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.579617:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e03e79 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.579637:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e03e79 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.579645:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e03e79 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760691.579797:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493616896:12345-192.168.203.60@tcp:4:dd.0 Request processed in 25975us (26194us total) trans 4294972304 rc 0/0 00000100:00100000:0.0:1789760691.579808:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5015 00000100:00100000:0.0:1789760691.579817:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701856136670 time 701894840850 state 0x2 ref 0 00000001:00080000:0.0:1789760691.579865:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972304 is committed 00000100:00100000:1.0:1789760691.583597:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760691.583675:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493617024 00000100:00100000:0.0:1789760691.583685:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493617024 00000100:00100000:0.0:1789760691.583736:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701856136670 time 701894840850 state 0x0 ref 1 00000100:00100000:0.0:1789760691.583751:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701895101266 time 701894840850 state 0x1 ref 1 00000100:00100000:0.0:1789760691.583781:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701895101266 time 701924816520 state 0x0 ref 1 00000100:00100000:0.0:1789760691.583794:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493617024:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760691.583842:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20422656, mtime: 0x6aad94b2, atime: 0x0, ctime: 0x6aad94b2, blocks: 0xa007 00010000:00010000:0.0:1789760691.583863:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03e80 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.583879:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760691.583883:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760691.583890:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03e80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.583914:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03e80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760691.624199:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20422656 -> 20426752 00002000:00010000:0.0:1789760691.624209:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760690 -> 1789760691 00002000:00010000:0.0:1789760691.624212:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760690 -> 1789760691 00010000:00010000:0.0:1789760691.624225:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03e80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.624239:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03e80 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.624248:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03e80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.624256:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760691.624262:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03e80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.624271:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03e80 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.624291:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03e80 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.624299:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e03e80 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760691.624468:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493617024:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40671us (40876us total) trans 4294972305 rc 0/0 00000100:00100000:0.0:1789760691.624478:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5016 00000100:00100000:0.0:1789760691.624488:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701895101266 time 701924816520 state 0x2 ref 0 00000100:00100000:1.0:1789760691.628173:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:0.0:1789760691.675277:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972306 is committed 00002000:00010000:2.0:1789760691.675530:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20426752 -> 20430848 00010000:00010000:2.0:1789760691.675545:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03e87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.675558:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03e87 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.675576:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03e87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.675584:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760691.675589:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03e87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.675597:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03e87 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.675617:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03e87 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.675625:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03e87 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760691.675720:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493617152:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47240us (47546us total) trans 4294972306 rc 0/0 00000100:00100000:2.0:1789760691.675731:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5017 00000100:00100000:2.0:1789760691.675742:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701925076936 time 701969499688 state 0x2 ref 0 00000100:00100000:1.0:1789760691.680109:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00002000:00010000:0.0:1789760691.728607:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20430848 -> 20434944 00010000:00010000:0.0:1789760691.728626:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e03e8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.728640:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e03e8e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.728650:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e03e8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.728663:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760691.728668:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e03e8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.728677:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e03e8e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.728697:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e03e8e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.728706:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e03e8e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760691.728915:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493617280:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48506us (48808us total) trans 4294972307 rc 0/0 00000100:00100000:0.0:1789760691.728926:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5018 00000100:00100000:0.0:1789760691.728945:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701969760104 time 702021419643 state 0x2 ref 0 00000100:00100000:0.0:1789760691.739698:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760691.743537:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 701969760104 time 702021419643 state 0x0 ref 1 00000100:00100000:2.0:1789760691.743554:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702021680059 time 702021419643 state 0x1 ref 1 00000100:00100000:2.0:1789760691.744196:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702021680059 time 702085228076 state 0x0 ref 1 00000100:00100000:2.0:1789760691.744213:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493617408:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760691.744270:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20434944, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa007 00010000:00010000:2.0:1789760691.744279:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e03e95 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.744296:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760691.744301:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760691.744308:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e03e95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760691.744323:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e03e95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760691.792500:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972308 is committed 00000100:00100000:0.0:1789760691.796897:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760691.797136:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493617536 00000100:00100000:1.0:1789760691.797147:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493617536 00000100:00100000:1.0:1789760691.797204:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702021680059 time 702085228076 state 0x0 ref 1 00000100:00100000:1.0:1789760691.797219:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702085488492 time 702085228076 state 0x1 ref 1 00000100:00100000:1.0:1789760691.797251:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702085488492 time 702138286081 state 0x0 ref 1 00000100:00100000:1.0:1789760691.797264:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493617536:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760691.797313:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20439040, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa007 00010000:00010000:1.0:1789760691.797322:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e03e9c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.797350:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760691.797355:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760691.797361:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e03e9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.797374:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e03e9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760691.835393:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972309 is committed 00002000:00010000:0.0:1789760691.835608:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20439040 -> 20443136 00010000:00010000:0.0:1789760691.835626:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e03e9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.835638:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e03e9c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.835647:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e03e9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.835654:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760691.835659:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e03e9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.835667:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e03e9c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.835686:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e03e9c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.835694:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e03e9c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760691.835795:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493617536:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38525us (38900us total) trans 4294972309 rc 0/0 00000100:00100000:0.0:1789760691.835805:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5020 00000100:00100000:0.0:1789760691.835816:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702085488492 time 702138286081 state 0x2 ref 0 00000100:00100000:1.0:1789760691.842566:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760691.843121:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493617664 00000100:00100000:0.0:1789760691.843492:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493617664 00000100:00100000:0.0:1789760691.843551:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702085488492 time 702138286081 state 0x0 ref 1 00000100:00100000:0.0:1789760691.843568:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702138546497 time 702138286081 state 0x1 ref 1 00000100:00100000:0.0:1789760691.843647:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702138546497 time 702184682190 state 0x0 ref 1 00000100:00100000:0.0:1789760691.843662:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493617664:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760691.843903:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20443136, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa007 00010000:00010000:0.0:1789760691.843914:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e03ea3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.843929:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760691.843934:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760691.843941:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e03ea3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.843954:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e03ea3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760691.894307:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972310 is committed 00002000:00010000:0.0:1789760691.894510:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20443136 -> 20447232 00010000:00010000:0.0:1789760691.894526:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e03ea3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.894538:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e03ea3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.894546:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e03ea3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.894553:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760691.894558:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e03ea3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.894568:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e03ea3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.894587:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e03ea3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.894596:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e03ea3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760691.894688:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493617664:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51025us (52128us total) trans 4294972310 rc 0/0 00000100:00100000:0.0:1789760691.894697:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5021 00000100:00100000:0.0:1789760691.894707:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702138546497 time 702184682190 state 0x2 ref 0 00000100:00100000:1.0:1789760691.900380:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760691.900503:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493617792 00000100:00100000:0.0:1789760691.900514:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493617792 00000100:00100000:0.0:1789760691.900565:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702138546497 time 702184682190 state 0x0 ref 1 00000100:00100000:0.0:1789760691.900578:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702184942606 time 702184682190 state 0x1 ref 1 00000100:00100000:0.0:1789760691.900606:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702184942606 time 702241641892 state 0x0 ref 1 00000100:00100000:0.0:1789760691.900618:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493617792:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760691.900663:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20447232, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa007 00010000:00010000:0.0:1789760691.900672:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e03eaa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.900687:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760691.900691:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760691.900697:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e03eaa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.900710:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e03eaa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760691.977499:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972311 is committed 00002000:00010000:1.0:1789760691.977598:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20447232 -> 20451328 00010000:00010000:1.0:1789760691.977615:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e03eaa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.977628:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e03eaa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.977637:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e03eaa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.977645:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760691.977650:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e03eaa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.977660:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e03eaa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.977679:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e03eaa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760691.977688:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e03eaa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760691.978046:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493617792:12345-192.168.203.60@tcp:4:dd.0 Request processed in 77424us (77669us total) trans 4294972311 rc 0/0 00000100:00100000:1.0:1789760691.978058:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5022 00000100:00100000:1.0:1789760691.978067:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702184942606 time 702241641892 state 0x2 ref 0 00000100:00100000:1.0:1789760691.983792:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760691.994693:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702184942606 time 702241641892 state 0x0 ref 1 00000100:00100000:0.0:1789760691.994712:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702241902308 time 702241641892 state 0x1 ref 1 00000100:00100000:0.0:1789760691.994800:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702241902308 time 702335834903 state 0x0 ref 1 00000100:00100000:0.0:1789760691.994816:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493617920:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760691.994878:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20451328, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa007 00010000:00010000:0.0:1789760691.994888:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e03eb1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.994990:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760691.994997:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760691.995003:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e03eb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760691.995023:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e03eb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760692.036627:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972312 is committed 00002000:00010000:0.0:1789760692.036726:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20451328 -> 20455424 00010000:00010000:0.0:1789760692.036741:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e03eb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.036754:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e03eb1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.036763:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e03eb1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.036770:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760692.036775:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e03eb1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.036796:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e03eb1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.036817:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e03eb1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.036826:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e03eb1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760692.036940:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493617920:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42124us (53152us total) trans 4294972312 rc 0/0 00000100:00100000:0.0:1789760692.036951:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5023 00000100:00100000:0.0:1789760692.037003:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702241902308 time 702335834903 state 0x2 ref 0 00000100:00100000:1.0:1789760692.041911:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760692.041999:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493618048 00000100:00100000:2.0:1789760692.042010:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493618048 00000100:00100000:2.0:1789760692.042065:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702241902308 time 702335834903 state 0x0 ref 1 00000100:00100000:2.0:1789760692.042079:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702336095319 time 702335834903 state 0x1 ref 1 00000100:00100000:2.0:1789760692.042110:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702336095319 time 702383144804 state 0x0 ref 1 00000100:00100000:2.0:1789760692.042123:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493618048:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760692.042174:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20455424, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa007 00010000:00010000:2.0:1789760692.042183:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03eb8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.042206:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760692.042211:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760692.042217:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03eb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.042231:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03eb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760692.090592:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760692.090613:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc5834a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3303:1876699519243264:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760692.090677:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1789760692.091114:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760692.091148:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc5834a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3303:1876699519243264:0@lo:13:osp-pre-0-0.0 00002000:00010000:1.0:1789760692.093108:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20455424 -> 20459520 00010000:00010000:1.0:1789760692.093123:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03eb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.093135:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03eb8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.093143:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03eb8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.093150:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760692.093155:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03eb8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.093165:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03eb8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.093188:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03eb8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.093196:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03eb8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760692.093278:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493618048:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51153us (51383us total) trans 4294972313 rc 0/0 00000100:00100000:1.0:1789760692.093288:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5024 00000100:00100000:1.0:1789760692.093296:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702336095319 time 702383144804 state 0x2 ref 0 00000100:00100000:1.0:1789760692.097282:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760692.097438:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493618176 00000100:00100000:2.0:1789760692.097448:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493618176 00000100:00100000:2.0:1789760692.097505:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702336095319 time 702383144804 state 0x0 ref 1 00000100:00100000:2.0:1789760692.097520:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702383405220 time 702383144804 state 0x1 ref 1 00000100:00100000:2.0:1789760692.097588:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702383405220 time 702438621976 state 0x0 ref 1 00000100:00100000:2.0:1789760692.097602:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493618176:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760692.097650:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20459520, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa007 00010000:00010000:2.0:1789760692.097660:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e03ebf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.097677:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760692.097682:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760692.097689:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e03ebf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.097703:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e03ebf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760692.166122:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760692.166232:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493618304 00000100:00100000:1.0:1789760692.166242:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493618304 00000100:00100000:1.0:1789760692.166304:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702383405220 time 702438621976 state 0x0 ref 1 00000100:00100000:1.0:1789760692.166320:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702438882392 time 702438621976 state 0x1 ref 1 00000100:00100000:1.0:1789760692.166355:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702438882392 time 702507389903 state 0x0 ref 1 00000100:00100000:1.0:1789760692.166368:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493618304:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760692.166416:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20463616, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa007 00010000:00010000:1.0:1789760692.166425:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03ec6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.166447:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760692.166452:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760692.166458:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03ec6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.166471:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03ec6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760692.212308:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20463616 -> 20467712 00010000:00010000:0.0:1789760692.212323:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03ec6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.212336:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03ec6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.212344:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03ec6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.212351:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760692.212356:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03ec6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.212366:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03ec6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.212386:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03ec6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.212394:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03ec6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760692.212491:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493618304:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46119us (46373us total) trans 4294972315 rc 0/0 00000100:00100000:0.0:1789760692.212501:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5026 00000100:00100000:0.0:1789760692.212511:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702438882392 time 702507389903 state 0x2 ref 0 00000100:00100000:0.0:1789760692.221666:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760692.224243:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493618432 00000100:00100000:2.0:1789760692.224258:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493618432 00000100:00100000:2.0:1789760692.224327:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702438882392 time 702507389903 state 0x0 ref 1 00000100:00100000:2.0:1789760692.224343:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702507650319 time 702507389903 state 0x1 ref 1 00000100:00100000:2.0:1789760692.224382:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702507650319 time 702565416874 state 0x0 ref 1 00000100:00100000:2.0:1789760692.224396:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493618432:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760692.224445:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20467712, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa007 00010000:00010000:2.0:1789760692.224454:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e03ecd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.224471:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760692.224476:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760692.224483:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e03ecd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.224496:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e03ecd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760692.321767:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972316 is committed 00000100:00100000:2.0:1789760692.329311:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493618560 00000100:00100000:2.0:1789760692.329320:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493618560 00000100:00100000:2.0:1789760692.329377:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702507650319 time 702565416874 state 0x0 ref 1 00000100:00100000:2.0:1789760692.329393:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702565677290 time 702565416874 state 0x1 ref 1 00000100:00100000:2.0:1789760692.329425:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702565677290 time 702670460886 state 0x0 ref 1 00000100:00100000:2.0:1789760692.329439:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493618560:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760692.329486:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20471808, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa007 00010000:00010000:2.0:1789760692.329496:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03ed4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.329633:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760692.329637:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760692.329644:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03ed4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.329658:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03ed4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760692.387447:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20471808 -> 20475904 00010000:00010000:2.0:1789760692.387462:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03ed4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.387482:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03ed4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.387491:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03ed4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.387499:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760692.387504:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03ed4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.387513:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03ed4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.387533:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03ed4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.387541:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03ed4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760692.387640:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493618560:12345-192.168.203.60@tcp:4:dd.0 Request processed in 58198us (58549us total) trans 4294972317 rc 0/0 00000100:00100000:2.0:1789760692.387651:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5028 00000100:00100000:2.0:1789760692.387661:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702565677290 time 702670460886 state 0x2 ref 0 00000100:00100000:1.0:1789760692.399435:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760692.404214:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702565677290 time 702670460886 state 0x0 ref 1 00000100:00100000:2.0:1789760692.404487:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702670721302 time 702670460886 state 0x1 ref 1 00000100:00100000:2.0:1789760692.404525:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702670721302 time 702745561195 state 0x0 ref 1 00000100:00100000:2.0:1789760692.404540:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493618688:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760692.404588:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20475904, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa007 00010000:00010000:2.0:1789760692.404598:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03edb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.404614:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760692.404618:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760692.404624:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03edb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.404637:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03edb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760692.458740:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20475904 -> 20480000 00010000:00010000:2.0:1789760692.458754:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03edb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.458766:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03edb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.458775:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03edb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.458789:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760692.458794:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03edb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.458802:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03edb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.458894:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03edb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.458983:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03edb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760692.459108:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493618688:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54567us (59737us total) trans 4294972318 rc 0/0 00000100:00100000:2.0:1789760692.459119:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5029 00000100:00100000:2.0:1789760692.459129:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702670721302 time 702745561195 state 0x2 ref 0 00000100:00100000:0.0:1789760692.466784:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760692.467073:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493618816 00000100:00100000:1.0:1789760692.467084:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493618816 00000100:00100000:1.0:1789760692.467144:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702670721302 time 702745561195 state 0x0 ref 1 00000100:00100000:1.0:1789760692.467159:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702745821611 time 702745561195 state 0x1 ref 1 00000100:00100000:1.0:1789760692.467191:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702745821611 time 702808226590 state 0x0 ref 1 00000100:00100000:1.0:1789760692.467203:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493618816:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760692.467262:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20480000, mtime: 0x6aad94b3, atime: 0x0, ctime: 0x6aad94b3, blocks: 0xa007 00010000:00010000:1.0:1789760692.467273:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e03ee2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.467292:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760692.467297:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760692.467303:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e03ee2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.467320:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e03ee2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760692.552447:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20480000 -> 20484096 00010000:00010000:0.0:1789760692.552468:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e03ee2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.552481:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e03ee2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.552489:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e03ee2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.552496:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760692.552501:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e03ee2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.552509:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e03ee2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.552526:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e03ee2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.552534:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e03ee2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760692.552626:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493618816:12345-192.168.203.60@tcp:4:dd.0 Request processed in 85420us (85847us total) trans 4294972319 rc 0/0 00000100:00100000:0.0:1789760692.552636:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5030 00000100:00100000:0.0:1789760692.552646:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702745821611 time 702808226590 state 0x2 ref 0 00000100:00100000:1.0:1789760692.557178:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760692.602224:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc1ec6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3306:1876699519243520:0@lo:400:kworker.0 00000100:00100000:0.0:1789760692.602293:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760692.602362:0:3306:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1789760692.602368:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4361880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3306:1876699519243392:0@lo:400:kworker.0 00000100:00100000:0.0:1789760692.602399:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760692.602420:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc452c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519243904:0@lo:400:kworker.0 00000100:00100000:0.0:1789760692.602443:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760692.602467:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc452df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3303:1876699519243648:0@lo:400:kworker.0 00000100:00100000:0.0:1789760692.602488:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760692.602506:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760692.602511:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc452d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3303:1876699519243776:0@lo:400:kworker.0 00000100:00100000:0.0:1789760692.602529:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760692.602556:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519243520 00000100:00100000:0.0:1789760692.602564:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519243520 00000100:00100000:0.0:1789760692.602601:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519243648 00000100:00100000:0.0:1789760692.602604:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519243648 00000100:00100000:0.0:1789760692.602626:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519243776 00000100:00100000:0.0:1789760692.602628:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519243776 00000100:00100000:0.0:1789760692.602646:0:9460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 434 00000100:00100000:0.0:1789760692.602654:0:9460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3306:x1876699519243520:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760692.602702:0:9460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3306:x1876699519243520:12345-0@lo:400:kworker.0 Request processed in 48us (408us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760692.602709:0:9460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 434 00000100:00100000:0.0:1789760692.602724:0:9460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 435 00000100:00100000:0.0:1789760692.602730:0:9460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876699519243648:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760692.602758:0:9460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876699519243648:12345-0@lo:400:kworker.0 Request processed in 28us (269us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760692.602765:0:9460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 435 00000100:00100000:0.0:1789760692.602775:0:9460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 436 00000100:00100000:0.0:1789760692.602781:0:9460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d08380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876699519243776:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760692.602804:0:9460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d08380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876699519243776:12345-0@lo:400:kworker.0 Request processed in 23us (274us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760692.602809:0:9460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 436 00000100:00100000:0.0:1789760692.602824:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519243392 00000100:00100000:0.0:1789760692.602827:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519243392 00000100:00100000:0.0:1789760692.602865:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 380 00000100:00100000:0.0:1789760692.602872:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7bfdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3306:x1876699519243392:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760692.602911:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7bfdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3306:x1876699519243392:12345-0@lo:400:kworker.0 Request processed in 40us (512us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760692.602918:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 380 00000100:00080000:0.0:1789760692.602951:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760692.603009:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc452d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3303:1876699519243776:0@lo:400:kworker.0 00000100:00100000:0.0:1789760692.603020:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc452df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3303:1876699519243648:0@lo:400:kworker.0 00000100:00100000:0.0:1789760692.603044:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4361880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3306:1876699519243392:0@lo:400:kworker.0 00000100:00100000:0.0:1789760692.603053:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc1ec6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3306:1876699519243520:0@lo:400:kworker.0 00000100:00100000:1.0:1789760692.603352:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519243904 00000100:00100000:1.0:1789760692.603357:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519243904 00000100:00100000:1.0:1789760692.603405:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 159 00000100:00100000:1.0:1789760692.603414:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519243904:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760692.603459:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519243904:12345-0@lo:400:kworker.0 Request processed in 47us (1015us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760692.603466:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 159 00000100:00100000:1.0:1789760692.603498:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc452c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519243904:0@lo:400:kworker.0 00000100:00080000:0.0:1789760692.603599:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00010000:1.0:1789760692.605209:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20484096 -> 20488192 00010000:00010000:1.0:1789760692.605226:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03ee9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.605240:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03ee9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.605250:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03ee9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.605258:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760692.605262:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03ee9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.605278:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03ee9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.605297:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03ee9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760692.605305:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03ee9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760692.605394:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7aa7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493618944:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47751us (48217us total) trans 4294972320 rc 0/0 00000100:00100000:1.0:1789760692.605404:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5031 00000100:00100000:1.0:1789760692.605414:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702808487006 time 702898663051 state 0x2 ref 0 00000100:00100000:1.0:1789760692.609094:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:2.0:1789760692.615439:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972320 is committed 00000001:00080000:0.0:1789760692.643487:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972321 is committed 00002000:00010000:2.0:1789760692.643589:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20488192 -> 20492288 00002000:00010000:2.0:1789760692.643594:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760691 -> 1789760692 00002000:00010000:2.0:1789760692.643597:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760691 -> 1789760692 00010000:00010000:2.0:1789760692.643610:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e03ef0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.643631:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e03ef0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.643642:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e03ef0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.643650:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760692.643655:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e03ef0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.643666:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e03ef0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.643686:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e03ef0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.643695:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e03ef0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760692.643819:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8794700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493619072:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34501us (34729us total) trans 4294972321 rc 0/0 00000100:00100000:2.0:1789760692.643831:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5032 00000100:00100000:2.0:1789760692.643852:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702898923467 time 702950337119 state 0x2 ref 0 00000100:00100000:0.0:1789760692.648676:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760692.649158:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493619200 00000100:00100000:2.0:1789760692.649266:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493619200 00000100:00100000:2.0:1789760692.649395:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702898923467 time 702950337119 state 0x0 ref 1 00000100:00100000:2.0:1789760692.649413:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702950597535 time 702950337119 state 0x1 ref 1 00000100:00100000:2.0:1789760692.649485:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702950597535 time 702990520677 state 0x0 ref 1 00000100:00100000:2.0:1789760692.649498:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493619200:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760692.649541:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20492288, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0xa007 00010000:00010000:2.0:1789760692.649549:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03ef7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.649566:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760692.649570:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760692.649576:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03ef7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.649590:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03ef7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760692.679762:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972322 is committed 00002000:00010000:2.0:1789760692.680077:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20492288 -> 20496384 00010000:00010000:2.0:1789760692.680093:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03ef7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.680108:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03ef7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.680124:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03ef7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.680132:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760692.680137:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03ef7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.680146:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03ef7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.680169:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03ef7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.680177:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03ef7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760692.680405:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493619200:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30904us (31734us total) trans 4294972322 rc 0/0 00000100:00100000:2.0:1789760692.680417:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5033 00000100:00100000:2.0:1789760692.680427:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702950597535 time 702990520677 state 0x2 ref 0 00000100:00100000:1.0:1789760692.686319:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760692.686475:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493619328 00000100:00100000:0.0:1789760692.686486:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493619328 00000100:00100000:0.0:1789760692.686541:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702950597535 time 702990520677 state 0x0 ref 1 00000100:00100000:0.0:1789760692.686555:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702990781093 time 702990520677 state 0x1 ref 1 00000100:00100000:0.0:1789760692.690176:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702990781093 time 703027925370 state 0x0 ref 1 00000100:00100000:0.0:1789760692.690220:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8796a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493619328:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760692.690281:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20496384, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0xa007 00010000:00010000:0.0:1789760692.690291:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03efe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.690314:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760692.690376:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760692.690385:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03efe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.690401:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03efe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760692.731527:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760692.740882:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972323 is committed 00002000:00010000:2.0:1789760692.741510:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20496384 -> 20500480 00010000:00010000:2.0:1789760692.741528:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03efe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.741546:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03efe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.741555:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03efe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.741563:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760692.741567:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03efe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.741577:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03efe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.741597:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03efe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.741605:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03efe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760692.741784:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8796a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493619328:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51577us (55467us total) trans 4294972323 rc 0/0 00000100:00100000:2.0:1789760692.741796:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5034 00000100:00100000:2.0:1789760692.741806:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 702990781093 time 703027925370 state 0x2 ref 0 00000100:00100000:1.0:1789760692.746230:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:1.0:1789760692.799857:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972324 is committed 00002000:00010000:2.0:1789760692.800255:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20500480 -> 20504576 00010000:00010000:2.0:1789760692.800272:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e03f05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.800285:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e03f05 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.800293:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e03f05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.800301:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760692.800305:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e03f05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.800313:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e03f05 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.800410:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e03f05 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.800418:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e03f05 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760692.800556:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c89f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493619456:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53999us (54330us total) trans 4294972324 rc 0/0 00000100:00100000:2.0:1789760692.800567:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5035 00000100:00100000:2.0:1789760692.800576:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 703028185786 time 703087574291 state 0x2 ref 0 00000100:00100000:2.0:1789760692.806126:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:0.0:1789760692.903245:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972325 is committed 00000100:00100000:1.0:1789760692.918558:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760692.918752:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493619712 00000100:00100000:2.0:1789760692.918762:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493619712 00000100:00100000:2.0:1789760692.918817:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 703087834707 time 703147436665 state 0x0 ref 1 00000100:00100000:2.0:1789760692.918833:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 703147697081 time 703147436665 state 0x1 ref 1 00000100:00100000:2.0:1789760692.918875:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 703147697081 time 703259910448 state 0x0 ref 1 00000100:00100000:2.0:1789760692.918889:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c89f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493619712:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760692.918956:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20508672, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0xa007 00010000:00010000:2.0:1789760692.919005:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03f13 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.919024:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760692.919029:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760692.919037:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03f13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.919052:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e03f13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760692.988202:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972326 is committed 00000100:00100000:0.0:1789760692.994231:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760692.994401:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493619840 00000100:00100000:2.0:1789760692.994412:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493619840 00000100:00100000:2.0:1789760692.994467:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 703147697081 time 703259910448 state 0x0 ref 1 00000100:00100000:2.0:1789760692.994484:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 703260170864 time 703259910448 state 0x1 ref 1 00000100:00100000:2.0:1789760692.994549:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 703260170864 time 703335583030 state 0x0 ref 1 00000100:00100000:2.0:1789760692.994563:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d0a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493619840:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760692.994618:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20512768, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0xa007 00010000:00010000:2.0:1789760692.994628:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e03f1a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.994660:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760692.994665:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760692.994672:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e03f1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760692.994688:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e03f1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760693.055195:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972327 is committed 00002000:00010000:2.0:1789760693.056118:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20512768 -> 20516864 00010000:00010000:2.0:1789760693.056139:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e03f1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.056155:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e03f1a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.056165:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e03f1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.056180:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760693.056185:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e03f1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.056195:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e03f1a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.056215:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e03f1a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.056224:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e03f1a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760693.056447:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d0a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493619840:12345-192.168.203.60@tcp:4:dd.0 Request processed in 61879us (62219us total) trans 4294972327 rc 0/0 00000100:00100000:2.0:1789760693.056458:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5038 00000100:00100000:2.0:1789760693.056469:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 703260170864 time 703335583030 state 0x2 ref 0 00000100:00100000:2.0:1789760693.062447:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760693.062529:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493619968 00000100:00100000:1.0:1789760693.062540:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493619968 00000100:00100000:1.0:1789760693.062612:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 703260170864 time 703335583030 state 0x0 ref 1 00000100:00100000:1.0:1789760693.062628:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 703335843446 time 703335583030 state 0x1 ref 1 00000100:00100000:1.0:1789760693.062662:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 703335843446 time 703403697176 state 0x0 ref 1 00000100:00100000:1.0:1789760693.062675:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493619968:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760693.062733:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20516864, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0xa007 00010000:00010000:1.0:1789760693.062742:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e03f21 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.062758:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760693.062762:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760693.062768:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e03f21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.062783:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e03f21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760693.063292:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760693.114313:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789760693.114379:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4ba5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519244032:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760693.114452:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760693.114804:0:6691:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519244032 00000100:00100000:2.0:1789760693.114813:0:6691:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519244032 00000100:00100000:2.0:1789760693.114876:0:6691:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 181 00000100:00100000:2.0:1789760693.114886:0:6691:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519244032:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760693.114932:0:6691:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760693.115001:0:6691:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519244032:12345-0@lo:13:osp-pre-1-0.0 Request processed in 115us (549us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760693.115009:0:6691:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 181 00000100:00100000:0.0:1789760693.115155:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4ba5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519244032:0@lo:13:osp-pre-1-0.0 00000001:00080000:2.0:1789760693.165179:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972328 is committed 00000100:00100000:1.0:1789760693.197921:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760693.318203:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760693.409599:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493620352 00000100:00100000:2.0:1789760693.409609:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493620352 00000100:00100000:2.0:1789760693.409664:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 703539553118 time 703659649705 state 0x0 ref 1 00000100:00100000:2.0:1789760693.409680:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 703659910121 time 703659649705 state 0x1 ref 1 00000100:00100000:2.0:1789760693.409709:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 703659910121 time 703750745123 state 0x0 ref 1 00000100:00100000:2.0:1789760693.409722:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9621016d3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493620352:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760693.409768:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20529152, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0xa007 00010000:00010000:2.0:1789760693.409778:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e03f36 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.409795:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760693.409800:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760693.409807:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e03f36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.409821:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e03f36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760693.505206:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972331 is committed 00000100:00100000:1.0:1789760693.509079:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:2.0:1789760693.556746:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972332 is committed 00002000:00010000:0.0:1789760693.556828:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20533248 -> 20537344 00010000:00010000:0.0:1789760693.556852:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e03f3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.556869:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e03f3d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.556886:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e03f3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.557024:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760693.557030:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e03f3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.557040:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e03f3d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.557061:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e03f3d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.557069:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e03f3d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760693.557204:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493620480:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47900us (48140us total) trans 4294972332 rc 0/0 00000100:00100000:0.0:1789760693.557216:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5043 00000100:00100000:0.0:1789760693.557226:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 703751005539 time 703850323114 state 0x2 ref 0 00000100:00100000:0.0:1789760693.560329:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493620608 00000100:00100000:0.0:1789760693.560337:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493620608 00000100:00100000:0.0:1789760693.560383:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 703751005539 time 703850323114 state 0x0 ref 1 00000100:00100000:0.0:1789760693.560395:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 703850583530 time 703850323114 state 0x1 ref 1 00000100:00100000:0.0:1789760693.560423:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 703850583530 time 703901458323 state 0x0 ref 1 00000100:00100000:0.0:1789760693.560435:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493620608:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760693.560482:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20537344, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0xa007 00010000:00010000:0.0:1789760693.560491:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e03f44 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.560503:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760693.560506:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760693.560510:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e03f44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.560521:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e03f44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760693.586506:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20537344 -> 20541440 00000001:00080000:2.0:1789760693.586518:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972333 is committed 00010000:00010000:0.0:1789760693.586526:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e03f44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.586540:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e03f44 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.586550:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e03f44 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.586558:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760693.586564:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e03f44 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.586574:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e03f44 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.586592:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e03f44 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.586600:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e03f44 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760693.586688:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493620608:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26249us (26529us total) trans 4294972333 rc 0/0 00000100:00100000:0.0:1789760693.586699:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5044 00000100:00100000:0.0:1789760693.586710:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 703850583530 time 703901458323 state 0x2 ref 0 00000100:00100000:0.0:1789760693.591549:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760693.591678:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493620736 00000100:00100000:2.0:1789760693.591688:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493620736 00000100:00100000:2.0:1789760693.592227:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 703850583530 time 703901458323 state 0x0 ref 1 00000100:00100000:2.0:1789760693.592243:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 703901718739 time 703901458323 state 0x1 ref 1 00000100:00100000:2.0:1789760693.592329:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 703901718739 time 703933363342 state 0x0 ref 1 00000100:00100000:2.0:1789760693.592351:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493620736:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760693.592399:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20541440, mtime: 0x6aad94b4, atime: 0x0, ctime: 0x6aad94b4, blocks: 0xa007 00010000:00010000:2.0:1789760693.592408:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03f4b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.592424:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760693.592429:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760693.592436:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03f4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.592449:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e03f4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760693.628186:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972334 is committed 00000100:00100000:1.0:1789760693.631713:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:2.0:1789760693.676801:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972335 is committed 00000100:00100000:0.0:1789760693.690006:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:2.0:1789760693.763438:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972336 is committed 00002000:00010000:2.0:1789760693.763505:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20549632 -> 20553728 00010000:00010000:2.0:1789760693.763520:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e03f59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.763540:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e03f59 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.763548:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e03f59 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.763555:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760693.763560:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e03f59 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.763569:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e03f59 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.763590:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e03f59 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.763598:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e03f59 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760693.763705:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d0a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493620992:12345-192.168.203.60@tcp:4:dd.0 Request processed in 73283us (73752us total) trans 4294972336 rc 0/0 00000100:00100000:2.0:1789760693.763715:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5047 00000100:00100000:2.0:1789760693.763725:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 703973213784 time 704031442662 state 0x2 ref 0 00000100:00100000:2.0:1789760693.768086:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760693.768227:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493621120 00000100:00100000:0.0:1789760693.768237:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493621120 00000100:00100000:0.0:1789760693.768291:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 703973213784 time 704031442662 state 0x0 ref 1 00000100:00100000:0.0:1789760693.768306:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704031703078 time 704031442662 state 0x1 ref 1 00000100:00100000:0.0:1789760693.768334:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704031703078 time 704109369944 state 0x0 ref 1 00000100:00100000:0.0:1789760693.768345:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493621120:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760693.768394:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20553728, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa007 00010000:00010000:0.0:1789760693.768402:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03f60 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.768418:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760693.768422:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760693.768428:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03f60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.768441:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03f60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760693.838790:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972337 is committed 00002000:00010000:1.0:1789760693.839054:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20553728 -> 20557824 00010000:00010000:1.0:1789760693.839071:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03f60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.839084:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03f60 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.839093:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03f60 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.839102:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760693.839107:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03f60 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.839117:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03f60 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.839137:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03f60 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760693.839145:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e03f60 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760693.839242:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493621120:12345-192.168.203.60@tcp:4:dd.0 Request processed in 70892us (71161us total) trans 4294972337 rc 0/0 00000100:00100000:1.0:1789760693.839250:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5048 00000100:00100000:1.0:1789760693.839261:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704031703078 time 704109369944 state 0x2 ref 0 00000100:00100000:3.0:1789760693.852105:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760693.852540:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493621248 00000100:00100000:2.0:1789760693.852551:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493621248 00000100:00100000:2.0:1789760693.852606:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704031703078 time 704109369944 state 0x0 ref 1 00000100:00100000:2.0:1789760693.852620:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704109630360 time 704109369944 state 0x1 ref 1 00000100:00100000:2.0:1789760693.852658:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704109630360 time 704193692990 state 0x0 ref 1 00000100:00100000:2.0:1789760693.852669:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493621248:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760693.852718:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20557824, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa007 00010000:00010000:2.0:1789760693.852727:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e03f67 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.852742:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760693.852747:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760693.852754:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e03f67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.852767:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e03f67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760693.941212:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972338 is committed 00002000:00010000:2.0:1789760693.941467:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20557824 -> 20561920 00010000:00010000:2.0:1789760693.941482:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e03f67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.941495:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e03f67 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.941503:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e03f67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.941511:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760693.941516:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e03f67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.941524:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e03f67 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.941544:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e03f67 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760693.941552:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e03f67 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760693.941646:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493621248:12345-192.168.203.60@tcp:4:dd.0 Request processed in 88973us (89546us total) trans 4294972338 rc 0/0 00000100:00100000:2.0:1789760693.941656:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5049 00000100:00100000:2.0:1789760693.941665:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704109630360 time 704193692990 state 0x2 ref 0 00000100:00100000:0.0:1789760693.949147:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760693.950161:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493621376 00000100:00100000:3.0:1789760693.950174:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493621376 00000100:00100000:3.0:1789760693.950235:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704109630360 time 704193692990 state 0x0 ref 1 00000100:00100000:3.0:1789760693.950251:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704193953406 time 704193692990 state 0x1 ref 1 00000100:00100000:3.0:1789760693.950283:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704193953406 time 704291318815 state 0x0 ref 1 00000100:00100000:3.0:1789760693.950296:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493621376:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760693.950347:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20561920, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa007 00010000:00010000:3.0:1789760693.950357:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03f6e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760693.950372:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760693.950378:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760693.950384:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03f6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760693.950397:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03f6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760694.004560:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972339 is committed 00002000:00010000:3.0:1789760694.004671:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20561920 -> 20566016 00010000:00010000:3.0:1789760694.004686:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03f6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.004699:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03f6e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.004714:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03f6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.004722:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760694.004727:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03f6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.004737:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03f6e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.004757:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03f6e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.004765:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03f6e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760694.004866:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493621376:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54567us (55724us total) trans 4294972339 rc 0/0 00000100:00100000:3.0:1789760694.004877:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5050 00000100:00100000:3.0:1789760694.004886:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704193953406 time 704291318815 state 0x2 ref 0 00000100:00100000:0.0:1789760694.008215:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760694.008309:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493621504 00000100:00100000:2.0:1789760694.008319:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493621504 00000100:00100000:2.0:1789760694.008374:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704193953406 time 704291318815 state 0x0 ref 1 00000100:00100000:2.0:1789760694.008392:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704291579231 time 704291318815 state 0x1 ref 1 00000100:00100000:2.0:1789760694.008455:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704291579231 time 704349490257 state 0x0 ref 1 00000100:00100000:2.0:1789760694.008469:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493621504:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760694.008528:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20566016, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa007 00010000:00010000:2.0:1789760694.008537:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e03f75 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.008553:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760694.008557:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760694.008563:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e03f75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.008578:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e03f75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760694.066384:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972340 is committed 00002000:00010000:1.0:1789760694.066654:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20566016 -> 20570112 00010000:00010000:1.0:1789760694.066670:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e03f75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.066684:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e03f75 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.066699:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e03f75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.066707:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760694.066712:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e03f75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.066722:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e03f75 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.066742:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e03f75 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.066750:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e03f75 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760694.066849:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493621504:12345-192.168.203.60@tcp:4:dd.0 Request processed in 58378us (58639us total) trans 4294972340 rc 0/0 00000100:00100000:1.0:1789760694.066860:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5051 00000100:00100000:1.0:1789760694.066869:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704291579231 time 704349490257 state 0x2 ref 0 00000100:00100000:0.0:1789760694.073436:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760694.073561:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493621632 00000100:00100000:1.0:1789760694.073571:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493621632 00000100:00100000:1.0:1789760694.073626:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704291579231 time 704349490257 state 0x0 ref 1 00000100:00100000:1.0:1789760694.073639:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704349750673 time 704349490257 state 0x1 ref 1 00000100:00100000:1.0:1789760694.073670:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704349750673 time 704414705262 state 0x0 ref 1 00000100:00100000:1.0:1789760694.073682:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493621632:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760694.073731:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20570112, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa007 00010000:00010000:1.0:1789760694.073740:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e03f7c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.073757:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760694.073762:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760694.073768:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e03f7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.073782:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e03f7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760694.127699:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972341 is committed 00002000:00010000:2.0:1789760694.127952:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20570112 -> 20574208 00010000:00010000:2.0:1789760694.127990:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e03f7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.128003:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e03f7c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.128013:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e03f7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.128026:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760694.128031:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e03f7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.128041:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e03f7c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.128063:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e03f7c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.128070:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e03f7c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760694.128310:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493621632:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54624us (54876us total) trans 4294972341 rc 0/0 00000100:00100000:2.0:1789760694.128402:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5052 00000100:00100000:2.0:1789760694.128413:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704349750673 time 704414705262 state 0x2 ref 0 00000100:00100000:0.0:1789760694.136067:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760694.136155:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493621760 00000100:00100000:3.0:1789760694.136166:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493621760 00000100:00100000:3.0:1789760694.136292:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704349750673 time 704414705262 state 0x0 ref 1 00000100:00100000:3.0:1789760694.136310:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704414965678 time 704414705262 state 0x1 ref 1 00000100:00100000:3.0:1789760694.136454:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704414965678 time 704477488180 state 0x0 ref 1 00000100:00100000:3.0:1789760694.136467:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493621760:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760694.136518:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20574208, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa007 00010000:00010000:3.0:1789760694.136526:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03f83 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.136541:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760694.136545:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760694.136552:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03f83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.136566:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03f83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760694.184375:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20574208 -> 20578304 00010000:00010000:0.0:1789760694.184395:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03f83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.184409:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03f83 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.184418:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03f83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.184427:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760694.184432:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03f83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.184446:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03f83 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.184465:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03f83 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.184473:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e03f83 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760694.184720:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493621760:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48122us (48530us total) trans 4294972342 rc 0/0 00000100:00100000:0.0:1789760694.184730:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5053 00000100:00100000:0.0:1789760694.184738:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704414965678 time 704477488180 state 0x2 ref 0 00000001:00080000:0.0:1789760694.184773:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972342 is committed 00000100:00100000:0.0:1789760694.190289:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760694.190454:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493621888 00000100:00100000:3.0:1789760694.190463:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493621888 00000100:00100000:3.0:1789760694.190515:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704414965678 time 704477488180 state 0x0 ref 1 00000100:00100000:3.0:1789760694.190529:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704477748596 time 704477488180 state 0x1 ref 1 00000100:00100000:3.0:1789760694.190559:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704477748596 time 704531594219 state 0x0 ref 1 00000100:00100000:3.0:1789760694.190572:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4361880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493621888:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760694.190618:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20578304, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa007 00010000:00010000:3.0:1789760694.190629:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e03f8a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.190645:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760694.190650:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760694.190656:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e03f8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.190669:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e03f8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760694.255118:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972343 is committed 00002000:00010000:2.0:1789760694.255221:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20578304 -> 20582400 00010000:00010000:2.0:1789760694.255238:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e03f8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.255249:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e03f8a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.255261:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e03f8a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.255268:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760694.255272:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e03f8a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.255280:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e03f8a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.255298:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e03f8a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.255306:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e03f8a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760694.255407:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4361880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493621888:12345-192.168.203.60@tcp:4:dd.0 Request processed in 64831us (65120us total) trans 4294972343 rc 0/0 00000100:00100000:2.0:1789760694.255418:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5054 00000100:00100000:2.0:1789760694.255426:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704477748596 time 704531594219 state 0x2 ref 0 00000100:00100000:0.0:1789760694.259230:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760694.259305:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493622016 00000100:00100000:1.0:1789760694.259321:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493622016 00000100:00100000:1.0:1789760694.259373:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704477748596 time 704531594219 state 0x0 ref 1 00000100:00100000:1.0:1789760694.259388:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704531854635 time 704531594219 state 0x1 ref 1 00000100:00100000:1.0:1789760694.259415:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704531854635 time 704600451281 state 0x0 ref 1 00000100:00100000:1.0:1789760694.259428:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493622016:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760694.259474:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20582400, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa007 00010000:00010000:1.0:1789760694.259482:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e03f91 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.259497:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760694.259510:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760694.259516:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e03f91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.259531:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e03f91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760694.322290:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972344 is committed 00002000:00010000:2.0:1789760694.322422:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20582400 -> 20586496 00010000:00010000:2.0:1789760694.322438:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e03f91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.322457:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e03f91 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.322465:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e03f91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.322472:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760694.322476:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e03f91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.322485:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e03f91 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.322512:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e03f91 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.322520:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e03f91 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760694.322609:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493622016:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63179us (63386us total) trans 4294972344 rc 0/0 00000100:00100000:2.0:1789760694.322620:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5055 00000100:00100000:2.0:1789760694.322629:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704531854635 time 704600451281 state 0x2 ref 0 00000100:00100000:0.0:1789760694.327933:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760694.328359:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493622144 00000100:00100000:2.0:1789760694.328369:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493622144 00000100:00100000:2.0:1789760694.328427:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704531854635 time 704600451281 state 0x0 ref 1 00000100:00100000:2.0:1789760694.328442:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704600711697 time 704600451281 state 0x1 ref 1 00000100:00100000:2.0:1789760694.328477:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704600711697 time 704669512343 state 0x0 ref 1 00000100:00100000:2.0:1789760694.328490:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493622144:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760694.328545:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20586496, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa007 00010000:00010000:2.0:1789760694.328554:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e03f98 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.328571:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760694.328575:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760694.328581:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e03f98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.328594:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e03f98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760694.382044:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972345 is committed 00002000:00010000:0.0:1789760694.382113:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20586496 -> 20590592 00010000:00010000:0.0:1789760694.382127:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e03f98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.382139:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e03f98 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.382148:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e03f98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.382155:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760694.382160:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e03f98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.382168:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e03f98 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.382187:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e03f98 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.382195:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e03f98 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760694.382292:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493622144:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53798us (54362us total) trans 4294972345 rc 0/0 00000100:00100000:0.0:1789760694.382302:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5056 00000100:00100000:0.0:1789760694.382321:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704600711697 time 704669512343 state 0x2 ref 0 00000100:00100000:0.0:1789760694.386799:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760694.386930:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493622272 00000100:00100000:2.0:1789760694.386939:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493622272 00000100:00100000:2.0:1789760694.387167:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704600711697 time 704669512343 state 0x0 ref 1 00000100:00100000:2.0:1789760694.387182:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704669772759 time 704669512343 state 0x1 ref 1 00000100:00100000:1.0:1789760694.387332:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704669772759 time 704728365755 state 0x0 ref 1 00000100:00100000:1.0:1789760694.387349:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493622272:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760694.387394:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20590592, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa007 00010000:00010000:1.0:1789760694.387403:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e03f9f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.387418:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760694.387423:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760694.387429:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e03f9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.387444:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e03f9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760694.444661:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972346 is committed 00002000:00010000:2.0:1789760694.444848:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20590592 -> 20594688 00010000:00010000:2.0:1789760694.444863:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e03f9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.444876:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e03f9f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.444892:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e03f9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.444899:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760694.444904:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e03f9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.444911:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e03f9f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.444930:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e03f9f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.444937:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e03f9f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760694.445059:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493622272:12345-192.168.203.60@tcp:4:dd.0 Request processed in 57710us (58264us total) trans 4294972346 rc 0/0 00000100:00100000:2.0:1789760694.445069:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5057 00000100:00100000:2.0:1789760694.445078:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704669772759 time 704728365755 state 0x2 ref 0 00000100:00100000:2.0:1789760694.449481:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760694.449625:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493622400 00000100:00100000:3.0:1789760694.449636:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493622400 00000100:00100000:3.0:1789760694.449692:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704669772759 time 704728365755 state 0x0 ref 1 00000100:00100000:3.0:1789760694.449707:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704728626171 time 704728365755 state 0x1 ref 1 00000100:00100000:3.0:1789760694.449739:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704728626171 time 704790774486 state 0x0 ref 1 00000100:00100000:3.0:1789760694.449752:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493622400:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760694.449800:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20594688, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa007 00010000:00010000:3.0:1789760694.449809:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03fa6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.449825:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760694.449842:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760694.449848:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03fa6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.449863:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03fa6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760694.499037:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972347 is committed 00002000:00010000:3.0:1789760694.499135:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20594688 -> 20598784 00010000:00010000:3.0:1789760694.499150:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03fa6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.499162:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03fa6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.499171:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03fa6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.499178:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760694.499184:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03fa6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.499193:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03fa6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.499211:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03fa6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.499219:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03fa6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760694.499403:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493622400:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49648us (49927us total) trans 4294972347 rc 0/0 00000100:00100000:3.0:1789760694.499413:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5058 00000100:00100000:3.0:1789760694.499422:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704728626171 time 704790774486 state 0x2 ref 0 00000100:00100000:0.0:1789760694.502518:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760694.502655:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493622528 00000100:00100000:2.0:1789760694.502666:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493622528 00000100:00100000:2.0:1789760694.502717:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704728626171 time 704790774486 state 0x0 ref 1 00000100:00100000:2.0:1789760694.502732:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704791034902 time 704790774486 state 0x1 ref 1 00000100:00100000:2.0:1789760694.502801:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704791034902 time 704843835909 state 0x0 ref 1 00000100:00100000:2.0:1789760694.502814:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493622528:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760694.502868:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20598784, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa007 00010000:00010000:2.0:1789760694.502877:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e03fad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.502900:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760694.502905:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760694.502911:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e03fad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.502926:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e03fad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760694.538109:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20598784 -> 20602880 00010000:00010000:2.0:1789760694.538127:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e03fad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.538140:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e03fad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.538156:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e03fad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.538163:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760694.538168:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e03fad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.538177:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e03fad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.538194:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e03fad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.538202:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e03fad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760694.538295:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493622528:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35478us (35782us total) trans 4294972348 rc 0/0 00000100:00100000:2.0:1789760694.538304:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5059 00000100:00100000:2.0:1789760694.538412:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704791034902 time 704843835909 state 0x2 ref 0 00000001:00080000:3.0:1789760694.538481:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972348 is committed 00000100:00100000:0.0:1789760694.543387:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760694.543514:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493622656 00000100:00100000:2.0:1789760694.543525:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493622656 00000100:00100000:2.0:1789760694.543578:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704791034902 time 704843835909 state 0x0 ref 1 00000100:00100000:2.0:1789760694.543593:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704844096325 time 704843835909 state 0x1 ref 1 00000100:00100000:2.0:1789760694.543640:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704844096325 time 704884675607 state 0x0 ref 1 00000100:00100000:2.0:1789760694.543652:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493622656:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760694.543702:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20602880, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa007 00010000:00010000:2.0:1789760694.543711:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e03fb4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.543726:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760694.543730:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760694.543736:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e03fb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.543751:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e03fb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760694.591557:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972349 is committed 00002000:00010000:0.0:1789760694.591753:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20602880 -> 20606976 00010000:00010000:0.0:1789760694.591768:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e03fb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.591781:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e03fb4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.591791:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e03fb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.591798:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760694.591803:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e03fb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.591813:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e03fb4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.591842:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e03fb4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.591850:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e03fb4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760694.592174:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493622656:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48519us (48793us total) trans 4294972349 rc 0/0 00000100:00100000:2.0:1789760694.592719:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5060 00000100:00100000:2.0:1789760694.592729:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704844096325 time 704884675607 state 0x2 ref 0 00000100:00100000:0.0:1789760694.596387:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760694.596478:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493622784 00000100:00100000:1.0:1789760694.596487:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493622784 00000100:00100000:1.0:1789760694.596543:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704844096325 time 704884675607 state 0x0 ref 1 00000100:00100000:1.0:1789760694.596558:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704884936023 time 704884675607 state 0x1 ref 1 00000100:00100000:1.0:1789760694.596684:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704884936023 time 704937717967 state 0x0 ref 1 00000100:00100000:1.0:1789760694.596699:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493622784:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760694.596746:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20606976, mtime: 0x6aad94b5, atime: 0x0, ctime: 0x6aad94b5, blocks: 0xa007 00010000:00010000:1.0:1789760694.596754:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e03fbb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.596769:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760694.596773:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760694.596780:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e03fbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.596792:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e03fbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760694.633118:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972350 is committed 00002000:00010000:1.0:1789760694.633295:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20606976 -> 20611072 00002000:00010000:1.0:1789760694.633299:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760693 -> 1789760694 00002000:00010000:1.0:1789760694.633302:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760693 -> 1789760694 00010000:00010000:1.0:1789760694.633317:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e03fbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.633335:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e03fbb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.633343:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e03fbb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.633350:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760694.633355:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e03fbb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.633364:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e03fbb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.633389:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e03fbb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.633397:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e03fbb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760694.633503:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493622784:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36801us (37191us total) trans 4294972350 rc 0/0 00000100:00100000:1.0:1789760694.633514:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5061 00000100:00100000:1.0:1789760694.633526:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704884936023 time 704937717967 state 0x2 ref 0 00000100:00100000:0.0:1789760694.639348:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760694.639439:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493622912 00000100:00100000:3.0:1789760694.639449:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493622912 00000100:00100000:3.0:1789760694.639514:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704884936023 time 704937717967 state 0x0 ref 1 00000100:00100000:3.0:1789760694.639530:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704937978383 time 704937717967 state 0x1 ref 1 00000100:00100000:3.0:1789760694.639598:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704937978383 time 704980632763 state 0x0 ref 1 00000100:00100000:3.0:1789760694.639611:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493622912:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760694.639669:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20611072, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:3.0:1789760694.639678:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03fc2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.639694:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760694.639698:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760694.639704:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03fc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.639718:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03fc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760694.705425:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972351 is committed 00002000:00010000:2.0:1789760694.705532:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20611072 -> 20615168 00010000:00010000:2.0:1789760694.705549:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03fc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.705563:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03fc2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.705574:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03fc2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.705584:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760694.705589:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03fc2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.705599:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03fc2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.705621:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03fc2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.705630:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e03fc2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760694.705728:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493622912:12345-192.168.203.60@tcp:4:dd.0 Request processed in 66114us (66385us total) trans 4294972351 rc 0/0 00000100:00100000:2.0:1789760694.705739:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5062 00000100:00100000:2.0:1789760694.705749:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704937978383 time 704980632763 state 0x2 ref 0 00000100:00100000:0.0:1789760694.712656:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760694.712785:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493623040 00000100:00100000:3.0:1789760694.712796:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493623040 00000100:00100000:3.0:1789760694.712858:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704937978383 time 704980632763 state 0x0 ref 1 00000100:00100000:3.0:1789760694.712887:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704980893179 time 704980632763 state 0x1 ref 1 00000100:00100000:3.0:1789760694.712915:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704980893179 time 705053950563 state 0x0 ref 1 00000100:00100000:3.0:1789760694.712926:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493623040:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760694.713011:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20615168, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:3.0:1789760694.713020:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03fc9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.713035:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760694.713040:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760694.713045:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03fc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.713058:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03fc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760694.765361:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972352 is committed 00002000:00010000:0.0:1789760694.765544:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20615168 -> 20619264 00010000:00010000:0.0:1789760694.765559:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03fc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.765571:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03fc9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.765580:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03fc9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.765588:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760694.765592:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03fc9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.765602:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03fc9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.765622:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03fc9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.765630:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e03fc9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760694.765725:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493623040:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52794us (53073us total) trans 4294972352 rc 0/0 00000100:00100000:0.0:1789760694.765736:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5063 00000100:00100000:0.0:1789760694.765746:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704980893179 time 705053950563 state 0x2 ref 0 00000100:00100000:0.0:1789760694.770425:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760694.770620:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493623168 00000100:00100000:1.0:1789760694.770631:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493623168 00000100:00100000:1.0:1789760694.770693:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 704980893179 time 705053950563 state 0x0 ref 1 00000100:00100000:1.0:1789760694.770708:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705054210979 time 705053950563 state 0x1 ref 1 00000100:00100000:1.0:1789760694.770742:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705054210979 time 705111776668 state 0x0 ref 1 00000100:00100000:1.0:1789760694.770755:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493623168:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760694.770807:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20619264, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:1.0:1789760694.770815:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03fd0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.770840:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760694.770844:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760694.770851:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03fd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.770864:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03fd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.778093:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760694.798209:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972353 is committed 00002000:00010000:1.0:1789760694.798406:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20619264 -> 20623360 00010000:00010000:1.0:1789760694.798422:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03fd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.798434:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03fd0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.798445:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03fd0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.798453:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760694.798458:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03fd0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.798467:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03fd0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.798489:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03fd0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.798498:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e03fd0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760694.798602:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493623168:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27845us (28181us total) trans 4294972353 rc 0/0 00000100:00100000:1.0:1789760694.798614:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5064 00000100:00100000:1.0:1789760694.798624:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705054210979 time 705111776668 state 0x2 ref 0 00000100:00100000:3.0:1789760694.802740:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760694.802823:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493623296 00000100:00100000:2.0:1789760694.802847:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493623296 00000100:00100000:2.0:1789760694.802910:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705054210979 time 705111776668 state 0x0 ref 1 00000100:00100000:2.0:1789760694.802925:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705112037084 time 705111776668 state 0x1 ref 1 00000100:00100000:2.0:1789760694.802953:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705112037084 time 705143988310 state 0x0 ref 1 00000100:00100000:2.0:1789760694.802989:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493623296:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760694.803036:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20623360, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:2.0:1789760694.803044:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e03fd7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.803060:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760694.803065:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760694.803072:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e03fd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.803085:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e03fd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760694.843340:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972354 is committed 00002000:00010000:3.0:1789760694.843579:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20623360 -> 20627456 00010000:00010000:3.0:1789760694.843596:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e03fd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.843610:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e03fd7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.843619:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e03fd7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.843627:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760694.843632:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e03fd7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.843643:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e03fd7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.843664:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e03fd7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760694.843671:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e03fd7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760694.843770:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493623296:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40779us (41033us total) trans 4294972354 rc 0/0 00000100:00100000:3.0:1789760694.843781:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5065 00000100:00100000:3.0:1789760694.843791:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705112037084 time 705143988310 state 0x2 ref 0 00000100:00100000:0.0:1789760694.848461:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760694.849325:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493623424 00000100:00100000:2.0:1789760694.849338:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493623424 00000100:00100000:2.0:1789760694.849396:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705112037084 time 705143988310 state 0x0 ref 1 00000100:00100000:2.0:1789760694.849411:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705144248726 time 705143988310 state 0x1 ref 1 00000100:00100000:2.0:1789760694.849449:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705144248726 time 705190483495 state 0x0 ref 1 00000100:00100000:2.0:1789760694.849461:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493623424:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760694.849528:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20627456, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:2.0:1789760694.849538:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03fde lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.849556:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760694.849561:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760694.849568:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03fde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.849584:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03fde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760694.887131:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972355 is committed 00002000:00010000:2.0:1789760694.887322:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20627456 -> 20631552 00010000:00010000:2.0:1789760694.887338:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03fde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.887351:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03fde lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.887360:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03fde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.887367:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760694.887371:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03fde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.887381:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03fde lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.887403:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03fde lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.887410:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03fde lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760694.887509:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493623424:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38036us (39046us total) trans 4294972355 rc 0/0 00000100:00100000:2.0:1789760694.887520:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5066 00000100:00100000:2.0:1789760694.887530:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705144248726 time 705190483495 state 0x2 ref 0 00000100:00100000:2.0:1789760694.892848:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760694.892929:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493623552 00000100:00100000:0.0:1789760694.892938:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493623552 00000100:00100000:0.0:1789760694.893011:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705144248726 time 705190483495 state 0x0 ref 1 00000100:00100000:0.0:1789760694.893026:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705190743911 time 705190483495 state 0x1 ref 1 00000100:00100000:0.0:1789760694.893051:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705190743911 time 705234087200 state 0x0 ref 1 00000100:00100000:0.0:1789760694.893062:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493623552:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760694.893108:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20631552, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:0.0:1789760694.893117:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03fe5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.893131:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760694.893136:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760694.893143:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03fe5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.893157:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03fe5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760694.945999:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972356 is committed 00002000:00010000:0.0:1789760694.946168:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20631552 -> 20635648 00010000:00010000:0.0:1789760694.946183:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03fe5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.946196:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03fe5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.946205:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03fe5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.946212:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760694.946228:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03fe5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.946239:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03fe5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.946263:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03fe5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.946270:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e03fe5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760694.946380:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493623552:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53314us (53548us total) trans 4294972356 rc 0/0 00000100:00100000:0.0:1789760694.946390:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5067 00000100:00100000:0.0:1789760694.946400:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705190743911 time 705234087200 state 0x2 ref 0 00000100:00100000:0.0:1789760694.950737:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760694.952213:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493623680 00000100:00100000:1.0:1789760694.952592:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493623680 00000100:00100000:1.0:1789760694.952656:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705190743911 time 705234087200 state 0x0 ref 1 00000100:00100000:1.0:1789760694.952674:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705234347616 time 705234087200 state 0x1 ref 1 00000100:00100000:1.0:1789760694.952921:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705234347616 time 705293955389 state 0x0 ref 1 00000100:00100000:1.0:1789760694.952937:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493623680:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760694.953023:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20635648, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:1.0:1789760694.953034:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e03fec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.953053:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760694.953058:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760694.953064:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e03fec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760694.953080:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e03fec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760694.997900:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972357 is committed 00002000:00010000:2.0:1789760694.998180:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20635648 -> 20639744 00010000:00010000:2.0:1789760694.998196:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e03fec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.998209:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e03fec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.998231:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e03fec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.998239:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760694.998244:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e03fec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.998260:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e03fec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.998282:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e03fec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760694.998290:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e03fec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760694.998403:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493623680:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45464us (47671us total) trans 4294972357 rc 0/0 00000100:00100000:2.0:1789760694.998414:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5068 00000100:00100000:2.0:1789760694.998424:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705234347616 time 705293955389 state 0x2 ref 0 00000100:00100000:1.0:1789760695.003162:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760695.003364:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493623808 00000100:00100000:3.0:1789760695.003374:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493623808 00000100:00100000:3.0:1789760695.003531:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705234347616 time 705293955389 state 0x0 ref 1 00000100:00100000:3.0:1789760695.003547:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705294215805 time 705293955389 state 0x1 ref 1 00000100:00100000:3.0:1789760695.003619:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705294215805 time 705344653669 state 0x0 ref 1 00000100:00100000:3.0:1789760695.003633:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493623808:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760695.003679:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20639744, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:3.0:1789760695.003688:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03ff3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.003704:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760695.003708:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760695.003714:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03ff3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.003729:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03ff3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760695.004907:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789760695.085355:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20639744 -> 20643840 00010000:00010000:2.0:1789760695.085375:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03ff3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.085388:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03ff3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.085396:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03ff3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.085403:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760695.085408:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03ff3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.085417:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03ff3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.085435:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03ff3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.085442:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e03ff3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760695.085545:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493623808:12345-192.168.203.60@tcp:4:dd.0 Request processed in 81910us (82387us total) trans 4294972358 rc 0/0 00000100:00100000:2.0:1789760695.085555:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5069 00000100:00100000:2.0:1789760695.085564:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705294215805 time 705344653669 state 0x2 ref 0 00000001:00080000:0.0:1789760695.089104:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972358 is committed 00000100:00100000:0.0:1789760695.090090:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760695.090183:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493623936 00000100:00100000:2.0:1789760695.090192:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493623936 00000100:00100000:2.0:1789760695.090378:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705294215805 time 705344653669 state 0x0 ref 1 00000100:00100000:2.0:1789760695.090394:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705344914085 time 705344653669 state 0x1 ref 1 00000100:00100000:2.0:1789760695.090485:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705344914085 time 705431519571 state 0x0 ref 1 00000100:00100000:2.0:1789760695.090500:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493623936:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760695.090551:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20643840, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:2.0:1789760695.090560:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03ffa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.090576:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760695.090580:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760695.090586:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03ffa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.090600:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03ffa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760695.198519:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972359 is committed 00002000:00010000:1.0:1789760695.198688:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20643840 -> 20647936 00010000:00010000:1.0:1789760695.198704:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03ffa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.198716:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03ffa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.198724:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03ffa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.198735:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760695.198740:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03ffa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.198748:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03ffa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.198767:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03ffa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.198774:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e03ffa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760695.204777:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493623936:12345-192.168.203.60@tcp:4:dd.0 Request processed in 108377us (108789us total) trans 4294972359 rc 0/0 00000100:00100000:1.0:1789760695.204795:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5070 00000100:00100000:1.0:1789760695.204807:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705344914085 time 705431519571 state 0x2 ref 0 00000100:00100000:1.0:1789760695.215119:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760695.215334:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493624064 00000100:00100000:3.0:1789760695.215344:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493624064 00000100:00100000:3.0:1789760695.215398:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705344914085 time 705431519571 state 0x0 ref 1 00000100:00100000:3.0:1789760695.215413:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705431779987 time 705431519571 state 0x1 ref 1 00000100:00100000:3.0:1789760695.215444:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705431779987 time 705556479192 state 0x0 ref 1 00000100:00100000:3.0:1789760695.215461:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493624064:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760695.215511:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20647936, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:3.0:1789760695.215521:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04001 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.215536:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760695.215541:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760695.215547:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04001 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.215562:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04001 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760695.293171:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972360 is committed 00002000:00010000:3.0:1789760695.293455:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20647936 -> 20652032 00010000:00010000:3.0:1789760695.293471:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04001 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.293484:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04001 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.293492:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04001 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.293500:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760695.293504:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04001 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.293514:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04001 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.293535:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04001 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.293543:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04001 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760695.293634:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493624064:12345-192.168.203.60@tcp:4:dd.0 Request processed in 78171us (78521us total) trans 4294972360 rc 0/0 00000100:00100000:3.0:1789760695.293646:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5071 00000100:00100000:3.0:1789760695.293656:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705431779987 time 705556479192 state 0x2 ref 0 00000100:00100000:0.0:1789760695.298189:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760695.298354:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493624192 00000100:00100000:1.0:1789760695.298363:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493624192 00000100:00100000:1.0:1789760695.298419:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705431779987 time 705556479192 state 0x0 ref 1 00000100:00100000:1.0:1789760695.298434:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705556739608 time 705556479192 state 0x1 ref 1 00000100:00100000:1.0:1789760695.298465:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705556739608 time 705639499987 state 0x0 ref 1 00000100:00100000:1.0:1789760695.298478:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493624192:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760695.298527:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20652032, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:1.0:1789760695.298536:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04008 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.298552:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760695.298557:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760695.298564:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04008 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.298577:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04008 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760695.382601:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972361 is committed 00002000:00010000:3.0:1789760695.385066:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20652032 -> 20656128 00010000:00010000:3.0:1789760695.385196:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04008 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.385211:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04008 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.385220:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04008 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.385229:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760695.385234:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04008 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.385243:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04008 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.385262:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04008 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.385270:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04008 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760695.386345:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493624192:12345-192.168.203.60@tcp:4:dd.0 Request processed in 87862us (88158us total) trans 4294972361 rc 0/0 00000100:00100000:3.0:1789760695.386358:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5072 00000100:00100000:3.0:1789760695.386369:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705556739608 time 705639499987 state 0x2 ref 0 00000100:00100000:0.0:1789760695.391405:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760695.391528:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493624320 00000100:00100000:3.0:1789760695.391537:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493624320 00000100:00100000:3.0:1789760695.391589:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705556739608 time 705639499987 state 0x0 ref 1 00000100:00100000:3.0:1789760695.391604:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705639760403 time 705639499987 state 0x1 ref 1 00000100:00100000:3.0:1789760695.391636:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705639760403 time 705732671698 state 0x0 ref 1 00000100:00100000:3.0:1789760695.391648:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493624320:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760695.391698:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20656128, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:3.0:1789760695.391707:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0400f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.391723:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760695.391727:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760695.391734:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0400f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.391748:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0400f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760695.452331:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972362 is committed 00002000:00010000:3.0:1789760695.452576:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20656128 -> 20660224 00010000:00010000:3.0:1789760695.452591:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0400f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.452603:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0400f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.452612:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0400f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.452619:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760695.452624:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0400f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.452632:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0400f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.452655:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0400f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.452664:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0400f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760695.452756:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493624320:12345-192.168.203.60@tcp:4:dd.0 Request processed in 61105us (61357us total) trans 4294972362 rc 0/0 00000100:00100000:3.0:1789760695.452766:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5073 00000100:00100000:3.0:1789760695.452776:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705639760403 time 705732671698 state 0x2 ref 0 00000100:00100000:1.0:1789760695.459591:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760695.459689:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493624448 00000100:00100000:0.0:1789760695.459700:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493624448 00000100:00100000:0.0:1789760695.459753:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705639760403 time 705732671698 state 0x0 ref 1 00000100:00100000:0.0:1789760695.459769:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705732932114 time 705732671698 state 0x1 ref 1 00000100:00100000:0.0:1789760695.459843:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705732932114 time 705800877895 state 0x0 ref 1 00000100:00100000:0.0:1789760695.459857:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620f39f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493624448:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760695.459918:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20660224, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:0.0:1789760695.459927:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04016 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.459944:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760695.459949:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760695.459955:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04016 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.460004:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04016 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760695.522556:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972363 is committed 00002000:00010000:0.0:1789760695.522685:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20660224 -> 20664320 00010000:00010000:0.0:1789760695.522700:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04016 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.522712:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04016 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.522720:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04016 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.522727:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760695.522732:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04016 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.522741:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04016 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.522762:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04016 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.522771:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04016 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760695.522872:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620f39f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493624448:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63012us (63286us total) trans 4294972363 rc 0/0 00000100:00100000:0.0:1789760695.522891:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5074 00000100:00100000:0.0:1789760695.522901:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705732932114 time 705800877895 state 0x2 ref 0 00000100:00100000:0.0:1789760695.528285:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760695.528443:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493624576 00000100:00100000:1.0:1789760695.528453:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493624576 00000100:00100000:1.0:1789760695.528505:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705732932114 time 705800877895 state 0x0 ref 1 00000100:00100000:1.0:1789760695.528519:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705801138311 time 705800877895 state 0x1 ref 1 00000100:00100000:1.0:1789760695.528580:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705801138311 time 705869614775 state 0x0 ref 1 00000100:00100000:1.0:1789760695.528593:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493624576:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760695.528641:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20664320, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:1.0:1789760695.528650:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0401d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.528664:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760695.528669:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760695.528674:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0401d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.528688:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0401d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760695.579275:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972364 is committed 00002000:00010000:1.0:1789760695.579489:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20664320 -> 20668416 00010000:00010000:1.0:1789760695.579506:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0401d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.579518:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0401d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.579527:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0401d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.579535:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760695.579540:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0401d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.579551:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0401d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.579575:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0401d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.579583:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0401d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760695.579683:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493624576:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51087us (51403us total) trans 4294972364 rc 0/0 00000100:00100000:1.0:1789760695.579695:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5075 00000100:00100000:1.0:1789760695.579706:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705801138311 time 705869614775 state 0x2 ref 0 00000100:00100000:1.0:1789760695.583936:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760695.584170:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493624704 00000100:00100000:3.0:1789760695.584179:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493624704 00000100:00100000:3.0:1789760695.584240:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705801138311 time 705869614775 state 0x0 ref 1 00000100:00100000:3.0:1789760695.584256:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705869875191 time 705869614775 state 0x1 ref 1 00000100:00100000:3.0:1789760695.584288:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705869875191 time 705925323521 state 0x0 ref 1 00000100:00100000:3.0:1789760695.584313:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8555500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493624704:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760695.584366:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20668416, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0xa007 00010000:00010000:3.0:1789760695.584376:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04024 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.584390:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760695.584395:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760695.584401:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04024 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.584415:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04024 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760695.617223:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20668416 -> 20672512 00002000:00010000:1.0:1789760695.617232:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760694 -> 1789760695 00002000:00010000:1.0:1789760695.617235:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760694 -> 1789760695 00010000:00010000:1.0:1789760695.617248:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04024 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.617261:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04024 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.617270:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04024 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.617340:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760695.617345:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04024 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.617354:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04024 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.617373:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04024 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.617380:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04024 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760695.617474:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8555500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493624704:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33170us (33542us total) trans 4294972365 rc 0/0 00000100:00100000:1.0:1789760695.617485:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5076 00000100:00100000:1.0:1789760695.617495:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705869875191 time 705925323521 state 0x2 ref 0 00000001:00080000:0.0:1789760695.618576:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972365 is committed 00000100:00100000:1.0:1789760695.620141:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760695.621125:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493624832 00000100:00100000:3.0:1789760695.621135:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493624832 00000100:00100000:3.0:1789760695.621190:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705869875191 time 705925323521 state 0x0 ref 1 00000100:00100000:3.0:1789760695.621206:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705925583937 time 705925323521 state 0x1 ref 1 00000100:00100000:3.0:1789760695.621264:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705925583937 time 705962299136 state 0x0 ref 1 00000100:00100000:3.0:1789760695.621287:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493624832:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760695.621335:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20672512, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:3.0:1789760695.621343:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0402b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.621358:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760695.621362:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760695.621369:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0402b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.621381:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0402b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760695.669812:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972366 is committed 00002000:00010000:2.0:1789760695.669942:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20672512 -> 20676608 00010000:00010000:2.0:1789760695.669956:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0402b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.669999:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0402b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.670009:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0402b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.670017:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760695.670023:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0402b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.670033:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0402b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.670056:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0402b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.670065:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0402b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760695.670149:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493624832:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48870us (50012us total) trans 4294972366 rc 0/0 00000100:00100000:2.0:1789760695.670159:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5077 00000100:00100000:2.0:1789760695.670168:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705925583937 time 705962299136 state 0x2 ref 0 00000100:00100000:3.0:1789760695.680151:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760695.680393:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493624960 00000100:00100000:0.0:1789760695.680404:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493624960 00000100:00100000:0.0:1789760695.680459:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705925583937 time 705962299136 state 0x0 ref 1 00000100:00100000:0.0:1789760695.680476:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705962559552 time 705962299136 state 0x1 ref 1 00000100:00100000:0.0:1789760695.680507:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705962559552 time 706021542496 state 0x0 ref 1 00000100:00100000:0.0:1789760695.680521:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493624960:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760695.680573:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20676608, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:0.0:1789760695.680583:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04032 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.680601:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760695.680607:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760695.680614:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04032 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.680639:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04032 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760695.750450:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972367 is committed 00002000:00010000:3.0:1789760695.750665:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20676608 -> 20680704 00010000:00010000:3.0:1789760695.750835:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04032 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.750849:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04032 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.750859:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04032 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.750867:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760695.750879:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04032 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.750889:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04032 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.750908:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04032 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.750917:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04032 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760695.751168:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493624960:12345-192.168.203.60@tcp:4:dd.0 Request processed in 70644us (71021us total) trans 4294972367 rc 0/0 00000100:00100000:3.0:1789760695.751179:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5078 00000100:00100000:3.0:1789760695.751191:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705962559552 time 706021542496 state 0x2 ref 0 00000100:00100000:3.0:1789760695.766684:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760695.767009:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493625088 00000100:00100000:1.0:1789760695.767019:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493625088 00000100:00100000:1.0:1789760695.767072:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 705962559552 time 706021542496 state 0x0 ref 1 00000100:00100000:1.0:1789760695.767090:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706021802912 time 706021542496 state 0x1 ref 1 00000100:00100000:2.0:1789760695.767156:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706021802912 time 706108189157 state 0x0 ref 1 00000100:00100000:2.0:1789760695.767173:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493625088:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760695.767232:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20680704, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:2.0:1789760695.767241:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04039 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.767258:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760695.767263:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760695.767270:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04039 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.767285:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04039 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760695.809157:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760695.812804:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972368 is committed 00002000:00010000:3.0:1789760695.812941:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20680704 -> 20684800 00010000:00010000:3.0:1789760695.812955:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04039 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.813000:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04039 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.813008:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04039 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.813015:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760695.813019:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04039 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.813027:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04039 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.813043:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04039 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.813056:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04039 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760695.813135:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493625088:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45962us (46456us total) trans 4294972368 rc 0/0 00000100:00100000:3.0:1789760695.813145:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5079 00000100:00100000:3.0:1789760695.813155:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706021802912 time 706108189157 state 0x2 ref 0 00000100:00100000:1.0:1789760695.819691:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760695.820003:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493625216 00000100:00100000:3.0:1789760695.820014:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493625216 00000100:00100000:3.0:1789760695.820071:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706021802912 time 706108189157 state 0x0 ref 1 00000100:00100000:3.0:1789760695.820085:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706108449573 time 706108189157 state 0x1 ref 1 00000100:00100000:3.0:1789760695.820106:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706108449573 time 706161142126 state 0x0 ref 1 00000100:00100000:3.0:1789760695.820118:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8557b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493625216:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760695.820165:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20684800, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:3.0:1789760695.820173:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04040 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.820188:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760695.820192:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760695.820198:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04040 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.820633:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04040 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760695.868674:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972369 is committed 00002000:00010000:3.0:1789760695.868787:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20684800 -> 20688896 00010000:00010000:3.0:1789760695.868801:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04040 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.868812:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04040 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.868832:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04040 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.868840:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760695.868844:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04040 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.868853:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04040 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.868881:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04040 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.868891:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04040 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760695.869021:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8557b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493625216:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48900us (49335us total) trans 4294972369 rc 0/0 00000100:00100000:3.0:1789760695.869031:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5080 00000100:00100000:3.0:1789760695.869040:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706108449573 time 706161142126 state 0x2 ref 0 00000100:00100000:1.0:1789760695.875919:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760695.876168:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493625344 00000100:00100000:2.0:1789760695.876179:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493625344 00000100:00100000:2.0:1789760695.876234:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706108449573 time 706161142126 state 0x0 ref 1 00000100:00100000:2.0:1789760695.876251:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706161402542 time 706161142126 state 0x1 ref 1 00000100:00100000:2.0:1789760695.876325:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706161402542 time 706217360125 state 0x0 ref 1 00000100:00100000:2.0:1789760695.876339:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8555180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493625344:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760695.876388:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20688896, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:2.0:1789760695.876398:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04047 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.876414:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760695.876419:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760695.876426:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04047 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.876440:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04047 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760695.910472:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972370 is committed 00002000:00010000:2.0:1789760695.911064:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20688896 -> 20692992 00010000:00010000:2.0:1789760695.911080:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04047 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.911100:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04047 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.911110:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04047 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.911118:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760695.911122:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04047 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.911132:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04047 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.911150:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04047 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.911158:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04047 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760695.911287:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8555180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493625344:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34946us (35373us total) trans 4294972370 rc 0/0 00000100:00100000:2.0:1789760695.911297:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5081 00000100:00100000:2.0:1789760695.911315:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706161402542 time 706217360125 state 0x2 ref 0 00000100:00100000:3.0:1789760695.926726:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760695.926866:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493625472 00000100:00100000:0.0:1789760695.926887:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493625472 00000100:00100000:0.0:1789760695.926940:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706161402542 time 706217360125 state 0x0 ref 1 00000100:00100000:0.0:1789760695.926954:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706217620541 time 706217360125 state 0x1 ref 1 00000100:00100000:0.0:1789760695.927024:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706217620541 time 706268059482 state 0x0 ref 1 00000100:00100000:0.0:1789760695.927036:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493625472:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760695.927301:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20692992, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:2.0:1789760695.927311:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0404e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.927326:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760695.927330:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760695.927336:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0404e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760695.927349:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0404e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760695.970806:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972371 is committed 00002000:00010000:3.0:1789760695.970942:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20692992 -> 20697088 00010000:00010000:3.0:1789760695.970957:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0404e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.971000:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0404e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.971009:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0404e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.971016:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760695.971021:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0404e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.971030:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0404e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.971050:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0404e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.971064:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0404e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760695.971155:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493625472:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44116us (44434us total) trans 4294972371 rc 0/0 00000100:00100000:3.0:1789760695.971167:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5082 00000100:00100000:3.0:1789760695.971177:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706217620541 time 706268059482 state 0x2 ref 0 00000100:00100000:1.0:1789760695.975043:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760695.975165:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493625600 00000100:00100000:3.0:1789760695.975176:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493625600 00000100:00100000:3.0:1789760695.975225:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706217620541 time 706268059482 state 0x0 ref 1 00000100:00100000:3.0:1789760695.975241:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706268319898 time 706268059482 state 0x1 ref 1 00000100:00100000:3.0:1789760695.975271:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706268319898 time 706316306649 state 0x0 ref 1 00000100:00100000:3.0:1789760695.975284:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8556a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493625600:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760695.975339:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20697088, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:3.0:1789760695.975348:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04055 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.975363:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760695.975368:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760695.975375:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04055 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760695.975392:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04055 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760696.020884:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972372 is committed 00002000:00010000:3.0:1789760696.021086:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20697088 -> 20701184 00010000:00010000:3.0:1789760696.021101:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04055 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.021114:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04055 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.021124:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04055 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.021131:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760696.021136:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04055 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.021146:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04055 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.021168:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04055 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.021176:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04055 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760696.021273:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8556a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493625600:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45986us (46235us total) trans 4294972372 rc 0/0 00000100:00100000:3.0:1789760696.021284:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5083 00000100:00100000:3.0:1789760696.021294:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706268319898 time 706316306649 state 0x2 ref 0 00000100:00100000:1.0:1789760696.025269:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760696.025619:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493625728 00000100:00100000:3.0:1789760696.025630:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493625728 00000100:00100000:3.0:1789760696.025681:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706268319898 time 706316306649 state 0x0 ref 1 00000100:00100000:3.0:1789760696.025698:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706316567065 time 706316306649 state 0x1 ref 1 00000100:00100000:3.0:1789760696.025764:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706316567065 time 706366798645 state 0x0 ref 1 00000100:00100000:3.0:1789760696.025777:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493625728:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760696.025832:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20701184, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:3.0:1789760696.025841:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0405c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.025857:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760696.025862:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760696.025869:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0405c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.025895:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0405c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760696.078703:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972373 is committed 00002000:00010000:1.0:1789760696.078841:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20701184 -> 20705280 00010000:00010000:1.0:1789760696.078856:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0405c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.078880:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0405c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.078889:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0405c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.078897:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760696.078901:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0405c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.078911:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0405c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.078931:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0405c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.078939:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0405c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760696.079063:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493625728:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53283us (53798us total) trans 4294972373 rc 0/0 00000100:00100000:1.0:1789760696.079073:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5084 00000100:00100000:1.0:1789760696.079083:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706316567065 time 706366798645 state 0x2 ref 0 00000100:00100000:0.0:1789760696.082923:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760696.083033:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493625856 00000100:00100000:1.0:1789760696.083042:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493625856 00000100:00100000:1.0:1789760696.083091:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706316567065 time 706366798645 state 0x0 ref 1 00000100:00100000:1.0:1789760696.083105:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706367059061 time 706366798645 state 0x1 ref 1 00000100:00100000:1.0:1789760696.083130:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706367059061 time 706424166417 state 0x0 ref 1 00000100:00100000:1.0:1789760696.083141:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493625856:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760696.083184:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20705280, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:1.0:1789760696.083193:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04063 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.083217:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760696.083222:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760696.083228:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04063 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.083242:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04063 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760696.116124:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972374 is committed 00002000:00010000:3.0:1789760696.116219:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20705280 -> 20709376 00010000:00010000:3.0:1789760696.116235:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04063 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.116247:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04063 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.116255:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04063 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.116263:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760696.116268:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04063 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.116277:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04063 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.116409:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04063 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.116420:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04063 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760696.116553:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493625856:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33407us (33634us total) trans 4294972374 rc 0/0 00000100:00100000:3.0:1789760696.116568:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5085 00000100:00100000:3.0:1789760696.116577:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706367059061 time 706424166417 state 0x2 ref 0 00000100:00100000:0.0:1789760696.121463:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760696.121561:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493625984 00000100:00100000:2.0:1789760696.121571:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493625984 00000100:00100000:2.0:1789760696.121627:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706367059061 time 706424166417 state 0x0 ref 1 00000100:00100000:2.0:1789760696.121643:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706424426833 time 706424166417 state 0x1 ref 1 00000100:00100000:2.0:1789760696.121708:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706424426833 time 706462742953 state 0x0 ref 1 00000100:00100000:2.0:1789760696.121722:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493625984:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760696.121771:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20709376, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:2.0:1789760696.121780:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0406a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.121796:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760696.121800:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760696.121807:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0406a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.121834:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0406a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760696.170420:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972375 is committed 00002000:00010000:2.0:1789760696.170792:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20709376 -> 20713472 00010000:00010000:2.0:1789760696.170809:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0406a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.171061:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0406a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.171070:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0406a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.171078:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760696.171083:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0406a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.171092:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0406a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.171112:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0406a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.171120:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0406a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760696.171206:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493625984:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49483us (49748us total) trans 4294972375 rc 0/0 00000100:00100000:2.0:1789760696.171217:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5086 00000100:00100000:2.0:1789760696.171227:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706424426833 time 706462742953 state 0x2 ref 0 00000100:00100000:0.0:1789760696.175812:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760696.176312:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493626112 00000100:00100000:3.0:1789760696.176323:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493626112 00000100:00100000:3.0:1789760696.176379:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706424426833 time 706462742953 state 0x0 ref 1 00000100:00100000:3.0:1789760696.176394:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706463003369 time 706462742953 state 0x1 ref 1 00000100:00100000:3.0:1789760696.176426:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706463003369 time 706517461025 state 0x0 ref 1 00000100:00100000:3.0:1789760696.176440:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493626112:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760696.176739:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20713472, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:3.0:1789760696.176748:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04071 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.176764:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760696.176769:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760696.176774:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04071 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760696.176789:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04071 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760696.238205:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972376 is committed 00002000:00010000:1.0:1789760696.241262:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20713472 -> 20717568 00010000:00010000:1.0:1789760696.241282:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04071 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.241306:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04071 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.241316:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04071 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.241330:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760696.241335:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04071 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.241345:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04071 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.241366:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04071 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.241373:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04071 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760696.241733:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493626112:12345-192.168.203.60@tcp:4:dd.0 Request processed in 65290us (65925us total) trans 4294972376 rc 0/0 00000100:00100000:1.0:1789760696.241747:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5087 00000100:00100000:1.0:1789760696.241758:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706463003369 time 706517461025 state 0x2 ref 0 00000100:00100000:0.0:1789760696.254042:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760696.254136:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493626240 00000100:00100000:1.0:1789760696.254146:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493626240 00000100:00100000:1.0:1789760696.254204:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706463003369 time 706517461025 state 0x0 ref 1 00000100:00100000:1.0:1789760696.254220:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706517721441 time 706517461025 state 0x1 ref 1 00000100:00100000:1.0:1789760696.254253:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706517721441 time 706595288437 state 0x0 ref 1 00000100:00100000:1.0:1789760696.254265:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493626240:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760696.254314:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20717568, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:1.0:1789760696.254323:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04078 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.254338:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760696.254343:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760696.254349:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04078 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.254363:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04078 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760696.307575:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972377 is committed 00002000:00010000:2.0:1789760696.307799:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20717568 -> 20721664 00010000:00010000:2.0:1789760696.307814:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04078 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.307837:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04078 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.307846:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04078 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.307855:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760696.307860:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04078 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.307879:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04078 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.307898:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04078 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.307906:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04078 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760696.308025:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493626240:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53757us (53989us total) trans 4294972377 rc 0/0 00000100:00100000:2.0:1789760696.308036:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5088 00000100:00100000:2.0:1789760696.308045:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706517721441 time 706595288437 state 0x2 ref 0 00000100:00100000:0.0:1789760696.315937:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760696.316051:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493626368 00000100:00100000:2.0:1789760696.316063:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493626368 00000100:00100000:2.0:1789760696.316119:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706517721441 time 706595288437 state 0x0 ref 1 00000100:00100000:2.0:1789760696.316134:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706595548853 time 706595288437 state 0x1 ref 1 00000100:00100000:2.0:1789760696.316154:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706595548853 time 706657190410 state 0x0 ref 1 00000100:00100000:2.0:1789760696.316167:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493626368:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760696.316219:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20721664, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:2.0:1789760696.316228:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0407f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.316243:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760696.316248:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760696.316254:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0407f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.316274:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0407f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760696.394651:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972378 is committed 00002000:00010000:2.0:1789760696.394764:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20721664 -> 20725760 00010000:00010000:2.0:1789760696.394780:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0407f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.394793:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0407f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.394803:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0407f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.394811:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760696.394827:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0407f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.394838:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0407f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.394857:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0407f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.394866:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0407f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760696.395103:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493626368:12345-192.168.203.60@tcp:4:dd.0 Request processed in 78933us (79170us total) trans 4294972378 rc 0/0 00000100:00100000:2.0:1789760696.395114:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5089 00000100:00100000:2.0:1789760696.395123:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706595548853 time 706657190410 state 0x2 ref 0 00000100:00100000:0.0:1789760696.398666:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760696.398797:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493626496 00000100:00100000:2.0:1789760696.398807:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493626496 00000100:00100000:2.0:1789760696.398933:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706595548853 time 706657190410 state 0x0 ref 1 00000100:00100000:2.0:1789760696.398949:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706657450826 time 706657190410 state 0x1 ref 1 00000100:00100000:2.0:1789760696.399053:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706657450826 time 706740087242 state 0x0 ref 1 00000100:00100000:2.0:1789760696.399067:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493626496:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760696.399115:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20725760, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:2.0:1789760696.399124:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04086 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.399140:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760696.399145:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760696.399150:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04086 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.399165:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04086 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.429105:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760696.429151:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760696.429181:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760696.429268:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493626624 00000100:00100000:3.0:1789760696.429278:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493626624 00000100:00100000:3.0:1789760696.429342:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 381 00000100:00100000:3.0:1789760696.429352:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7bfc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2367:x1876699493626624:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:3.0:1789760696.429417:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7bfc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2367:x1876699493626624:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 67us (266us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760696.429424:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 381 00000100:00100000:2.0:1789760696.430048:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493626752 00000100:00100000:2.0:1789760696.430053:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493626752 00000100:00100000:2.0:1789760696.430102:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493626880 00000100:00100000:2.0:1789760696.430106:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493626880 00000100:00100000:2.0:1789760696.430128:0:9460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 437 00000100:00100000:2.0:1789760696.430135:0:9460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2368:x1876699493626752:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:2.0:1789760696.430175:0:9460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2368:x1876699493626752:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 41us (1080us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760696.430182:0:9460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 437 00000100:00100000:2.0:1789760696.430195:0:9460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 438 00000100:00100000:2.0:1789760696.430213:0:9460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2367:x1876699493626880:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:2.0:1789760696.430239:0:9460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2367:x1876699493626880:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 37us (1058us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760696.430245:0:9460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 438 00000100:00080000:3.0:1789760696.430748:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:1.0:1789760696.457881:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972379 is committed 00002000:00010000:2.0:1789760696.458163:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20725760 -> 20729856 00010000:00010000:2.0:1789760696.458180:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04086 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.458194:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04086 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.458215:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04086 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.458229:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760696.458234:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04086 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.458243:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04086 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.458263:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04086 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.458270:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04086 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760696.458481:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493626496:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59411us (59818us total) trans 4294972379 rc 0/0 00000100:00100000:2.0:1789760696.458548:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5090 00000100:00100000:2.0:1789760696.458559:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706657450826 time 706740087242 state 0x2 ref 0 00000100:00100000:3.0:1789760696.461774:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760696.463019:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493627008 00000100:00100000:1.0:1789760696.463029:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493627008 00000100:00100000:1.0:1789760696.463085:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706657450826 time 706740087242 state 0x0 ref 1 00000100:00100000:1.0:1789760696.463100:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706740347658 time 706740087242 state 0x1 ref 1 00000100:00100000:1.0:1789760696.463130:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706740347658 time 706804165852 state 0x0 ref 1 00000100:00100000:1.0:1789760696.463142:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493627008:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760696.463190:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20729856, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:1.0:1789760696.463200:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0408d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.463225:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760696.463230:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760696.463237:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0408d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.463250:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0408d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760696.509754:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972380 is committed 00002000:00010000:0.0:1789760696.510044:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20729856 -> 20733952 00010000:00010000:0.0:1789760696.510060:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0408d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.510073:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0408d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.510082:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0408d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.510090:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760696.510096:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0408d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.510105:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0408d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.510124:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0408d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.510132:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0408d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.510228:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493627008:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47083us (48459us total) trans 4294972380 rc 0/0 00000100:00100000:0.0:1789760696.510240:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5091 00000100:00100000:0.0:1789760696.510249:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706740347658 time 706804165852 state 0x2 ref 0 00000100:00100000:0.0:1789760696.513776:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760696.513885:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493627136 00000100:00100000:1.0:1789760696.513895:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493627136 00000100:00100000:1.0:1789760696.513993:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706740347658 time 706804165852 state 0x0 ref 1 00000100:00100000:1.0:1789760696.514010:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706804426268 time 706804165852 state 0x1 ref 1 00000100:00100000:1.0:1789760696.514034:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706804426268 time 706855069538 state 0x0 ref 1 00000100:00100000:1.0:1789760696.514048:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493627136:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760696.514094:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20733952, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:1.0:1789760696.514103:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04094 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.514117:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760696.514121:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760696.514126:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04094 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.514138:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04094 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760696.554898:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972381 is committed 00002000:00010000:2.0:1789760696.555078:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20733952 -> 20738048 00010000:00010000:2.0:1789760696.555093:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04094 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.555104:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04094 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.555118:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04094 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.555125:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760696.555130:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04094 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.555138:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04094 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.555156:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04094 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.555164:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04094 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760696.555349:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493627136:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41241us (41521us total) trans 4294972381 rc 0/0 00000100:00100000:2.0:1789760696.555363:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5092 00000100:00100000:2.0:1789760696.555373:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706804426268 time 706855069538 state 0x2 ref 0 00000100:00100000:3.0:1789760696.558089:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760696.558283:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493627264 00000100:00100000:1.0:1789760696.558293:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493627264 00000100:00100000:1.0:1789760696.558352:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706804426268 time 706855069538 state 0x0 ref 1 00000100:00100000:1.0:1789760696.558366:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706855329954 time 706855069538 state 0x1 ref 1 00000100:00100000:1.0:1789760696.558394:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706855329954 time 706899430011 state 0x0 ref 1 00000100:00100000:1.0:1789760696.558405:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493627264:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760696.558448:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20738048, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:1.0:1789760696.558457:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0409b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.558478:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760696.558482:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760696.558488:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0409b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.558502:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0409b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760696.587569:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972382 is committed 00002000:00010000:2.0:1789760696.587736:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20738048 -> 20742144 00010000:00010000:2.0:1789760696.587753:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0409b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.587772:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0409b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.587782:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0409b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.587790:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760696.587795:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0409b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.587805:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0409b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.587833:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0409b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.587843:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0409b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760696.588035:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493627264:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29626us (29947us total) trans 4294972382 rc 0/0 00000100:00100000:2.0:1789760696.588046:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5093 00000100:00100000:2.0:1789760696.588056:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706855329954 time 706899430011 state 0x2 ref 0 00000100:00100000:3.0:1789760696.590889:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760696.591226:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493627392 00000100:00100000:2.0:1789760696.591237:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493627392 00000100:00100000:2.0:1789760696.591291:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706855329954 time 706899430011 state 0x0 ref 1 00000100:00100000:2.0:1789760696.591369:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706899690427 time 706899430011 state 0x1 ref 1 00000100:00100000:2.0:1789760696.591396:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706899690427 time 706932431160 state 0x0 ref 1 00000100:00100000:2.0:1789760696.591410:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493627392:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760696.591456:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20742144, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0xa007 00010000:00010000:2.0:1789760696.591465:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040a2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.591480:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760696.591485:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760696.591491:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.591505:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760696.623124:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972383 is committed 00002000:00010000:2.0:1789760696.623207:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20742144 -> 20746240 00002000:00010000:2.0:1789760696.623212:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760695 -> 1789760696 00002000:00010000:2.0:1789760696.623215:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760695 -> 1789760696 00010000:00010000:2.0:1789760696.623228:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.623241:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040a2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.623252:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.623260:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760696.623266:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.623274:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040a2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.623364:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040a2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.623374:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040a2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760696.623559:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493627392:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32148us (32675us total) trans 4294972383 rc 0/0 00000100:00100000:2.0:1789760696.623570:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5094 00000100:00100000:2.0:1789760696.623581:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706899690427 time 706932431160 state 0x2 ref 0 00000100:00100000:0.0:1789760696.627178:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760696.627268:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493627520 00000100:00100000:2.0:1789760696.627277:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493627520 00000100:00100000:2.0:1789760696.627398:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706899690427 time 706932431160 state 0x0 ref 1 00000100:00100000:2.0:1789760696.627414:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706932691576 time 706932431160 state 0x1 ref 1 00000100:00100000:2.0:1789760696.627552:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706932691576 time 706968585700 state 0x0 ref 1 00000100:00100000:2.0:1789760696.627566:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493627520:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760696.627613:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20746240, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:2.0:1789760696.627621:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040a9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.627637:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760696.627641:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760696.627647:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.627660:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760696.676423:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20746240 -> 20750336 00010000:00010000:2.0:1789760696.676442:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.676515:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040a9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.676525:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.676533:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760696.676538:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.676548:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040a9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.676569:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040a9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.676575:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040a9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760696.676685:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493627520:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49117us (49511us total) trans 4294972384 rc 0/0 00000100:00100000:2.0:1789760696.676696:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5095 00000100:00100000:2.0:1789760696.676706:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706932691576 time 706968585700 state 0x2 ref 0 00000001:00080000:0.0:1789760696.676773:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972384 is committed 00000100:00100000:3.0:1789760696.680324:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760696.680387:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493627648 00000100:00100000:2.0:1789760696.680397:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493627648 00000100:00100000:2.0:1789760696.680460:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706932691576 time 706968585700 state 0x0 ref 1 00000100:00100000:2.0:1789760696.680477:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706968846116 time 706968585700 state 0x1 ref 1 00000100:00100000:2.0:1789760696.680512:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706968846116 time 707021547285 state 0x0 ref 1 00000100:00100000:2.0:1789760696.680525:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493627648:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760696.680572:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20750336, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:2.0:1789760696.680581:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040b0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.680597:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760696.680602:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760696.680609:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.680622:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760696.714460:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972385 is committed 00002000:00010000:2.0:1789760696.714625:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20750336 -> 20754432 00010000:00010000:2.0:1789760696.714641:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.714654:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040b0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.714663:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.714671:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760696.714676:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.714686:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040b0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.714711:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040b0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.714720:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040b0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760696.714944:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493627648:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34416us (34625us total) trans 4294972385 rc 0/0 00000100:00100000:2.0:1789760696.714955:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5096 00000100:00100000:2.0:1789760696.715003:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706968846116 time 707021547285 state 0x2 ref 0 00000100:00100000:0.0:1789760696.719406:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760696.719636:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493627776 00000100:00100000:1.0:1789760696.719647:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493627776 00000100:00100000:1.0:1789760696.719700:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 706968846116 time 707021547285 state 0x0 ref 1 00000100:00100000:1.0:1789760696.719715:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707021807701 time 707021547285 state 0x1 ref 1 00000100:00100000:1.0:1789760696.719781:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707021807701 time 707060815172 state 0x0 ref 1 00000100:00100000:1.0:1789760696.719795:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493627776:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760696.719852:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20754432, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:1.0:1789760696.719862:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040b7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.719888:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760696.719893:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760696.719898:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.719917:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760696.755878:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20754432 -> 20758528 00010000:00010000:2.0:1789760696.755899:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.755912:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040b7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.755923:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.755930:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760696.755935:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.755949:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040b7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.755996:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040b7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.756006:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040b7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760696.756093:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493627776:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36294us (36690us total) trans 4294972386 rc 0/0 00000100:00100000:2.0:1789760696.756104:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5097 00000100:00100000:2.0:1789760696.756114:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707021807701 time 707060815172 state 0x2 ref 0 00000100:00100000:2.0:1789760696.759428:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760696.759633:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493627904 00000100:00100000:2.0:1789760696.759642:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493627904 00000100:00100000:2.0:1789760696.759697:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707021807701 time 707060815172 state 0x0 ref 1 00000100:00100000:2.0:1789760696.759712:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707061075588 time 707060815172 state 0x1 ref 1 00000100:00100000:2.0:1789760696.759737:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707061075588 time 707100772086 state 0x0 ref 1 00000100:00100000:2.0:1789760696.759750:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493627904:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760696.759797:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20758528, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:2.0:1789760696.759806:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040be lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.759830:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760696.759834:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760696.759841:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.759854:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760696.760594:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972386 is committed 00000001:00080000:0.0:1789760696.792602:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972387 is committed 00002000:00010000:1.0:1789760696.792689:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20758528 -> 20762624 00010000:00010000:1.0:1789760696.792707:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.792721:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040be lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.792731:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.792739:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760696.792743:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.792752:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.792773:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040be lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.792780:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e040be lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760696.793015:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493627904:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33262us (33591us total) trans 4294972387 rc 0/0 00000100:00100000:1.0:1789760696.793026:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5098 00000100:00100000:1.0:1789760696.793037:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707061075588 time 707100772086 state 0x2 ref 0 00000100:00100000:0.0:1789760696.797928:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760696.798125:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493628032 00000100:00100000:2.0:1789760696.798135:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493628032 00000100:00100000:2.0:1789760696.798193:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707061075588 time 707100772086 state 0x0 ref 1 00000100:00100000:2.0:1789760696.798209:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707101032502 time 707100772086 state 0x1 ref 1 00000100:00100000:2.0:1789760696.798268:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707101032502 time 707139303006 state 0x0 ref 1 00000100:00100000:2.0:1789760696.798282:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9621016d1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493628032:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760696.798395:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20762624, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:2.0:1789760696.798404:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040c5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.798421:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760696.798426:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760696.798432:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.798448:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.827095:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760696.838354:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972388 is committed 00002000:00010000:2.0:1789760696.838573:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20762624 -> 20766720 00010000:00010000:2.0:1789760696.838589:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.838603:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040c5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.838612:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.838620:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760696.838625:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.838635:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040c5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.838654:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040c5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.838664:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040c5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760696.838826:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9621016d1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493628032:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40533us (40950us total) trans 4294972388 rc 0/0 00000100:00100000:2.0:1789760696.838838:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5099 00000100:00100000:2.0:1789760696.838849:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707101032502 time 707139303006 state 0x2 ref 0 00000100:00100000:3.0:1789760696.842762:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760696.843239:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493628160 00000100:00100000:2.0:1789760696.843249:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493628160 00000100:00100000:2.0:1789760696.843369:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707101032502 time 707139303006 state 0x0 ref 1 00000100:00100000:2.0:1789760696.843385:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707139563422 time 707139303006 state 0x1 ref 1 00000100:00100000:2.0:1789760696.843521:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707139563422 time 707184488055 state 0x0 ref 1 00000100:00100000:2.0:1789760696.843537:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493628160:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760696.843584:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20766720, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:2.0:1789760696.843594:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040cc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.843609:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760696.843614:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760696.843621:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.843635:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760696.882284:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972389 is committed 00002000:00010000:2.0:1789760696.882666:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20766720 -> 20770816 00010000:00010000:2.0:1789760696.882682:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.882696:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040cc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.882706:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.882713:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760696.882718:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.882726:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040cc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.882744:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040cc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.882753:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e040cc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760696.883738:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493628160:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40196us (40977us total) trans 4294972389 rc 0/0 00000100:00100000:2.0:1789760696.883754:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5100 00000100:00100000:2.0:1789760696.883764:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707139563422 time 707184488055 state 0x2 ref 0 00000100:00100000:2.0:1789760696.888215:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760696.888395:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493628288 00000100:00100000:1.0:1789760696.888404:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493628288 00000100:00100000:1.0:1789760696.888565:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707139563422 time 707184488055 state 0x0 ref 1 00000100:00100000:1.0:1789760696.888583:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707184748471 time 707184488055 state 0x1 ref 1 00000100:00100000:1.0:1789760696.888643:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707184748471 time 707229677684 state 0x0 ref 1 00000100:00100000:1.0:1789760696.888657:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493628288:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760696.888706:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20770816, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:1.0:1789760696.888714:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e040d3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.888730:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760696.888734:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760696.888740:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e040d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.888760:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e040d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760696.931427:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20770816 -> 20774912 00010000:00010000:1.0:1789760696.931446:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e040d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.931473:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e040d3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.931483:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e040d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.931491:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760696.931496:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e040d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.931504:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e040d3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.931522:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e040d3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.931531:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e040d3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760696.931635:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493628288:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42976us (43424us total) trans 4294972390 rc 0/0 00000100:00100000:1.0:1789760696.931645:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5101 00000100:00100000:1.0:1789760696.931653:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707184748471 time 707229677684 state 0x2 ref 0 00000001:00080000:1.0:1789760696.931691:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972390 is committed 00000100:00100000:2.0:1789760696.935936:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760696.936576:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493628416 00000100:00100000:1.0:1789760696.936587:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493628416 00000100:00100000:1.0:1789760696.936642:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707184748471 time 707229677684 state 0x0 ref 1 00000100:00100000:1.0:1789760696.936657:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707229938100 time 707229677684 state 0x1 ref 1 00000100:00100000:1.0:1789760696.936726:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707229938100 time 707277760381 state 0x0 ref 1 00000100:00100000:1.0:1789760696.936739:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493628416:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760696.936786:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20774912, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:1.0:1789760696.936795:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040da lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.936823:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760696.936828:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760696.936835:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.936849:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760696.970363:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20774912 -> 20779008 00010000:00010000:1.0:1789760696.970382:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.970395:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040da lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.970404:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.970411:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760696.970416:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.970426:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040da lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.970446:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040da lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760696.970616:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040da lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760696.970711:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493628416:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33969us (34780us total) trans 4294972391 rc 0/0 00000100:00100000:1.0:1789760696.970725:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5102 00000100:00100000:1.0:1789760696.970735:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707229938100 time 707277760381 state 0x2 ref 0 00000001:00080000:0.0:1789760696.970818:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972391 is committed 00000100:00100000:0.0:1789760696.976439:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760696.976609:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493628544 00000100:00100000:2.0:1789760696.976620:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493628544 00000100:00100000:2.0:1789760696.976677:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707229938100 time 707277760381 state 0x0 ref 1 00000100:00100000:2.0:1789760696.976694:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707278020797 time 707277760381 state 0x1 ref 1 00000100:00100000:2.0:1789760696.976765:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707278020797 time 707317799016 state 0x0 ref 1 00000100:00100000:2.0:1789760696.976780:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493628544:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760696.976837:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20779008, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:2.0:1789760696.976846:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e040e1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.976871:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760696.976876:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760696.976884:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e040e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760696.976898:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e040e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760697.017767:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20779008 -> 20783104 00010000:00010000:1.0:1789760697.017786:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e040e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.017803:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e040e1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.017821:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e040e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.017830:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760697.017834:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e040e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.017844:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e040e1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.017873:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e040e1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.017881:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e040e1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760697.017998:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493628544:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41217us (41565us total) trans 4294972392 rc 0/0 00000100:00100000:1.0:1789760697.018009:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5103 00000100:00100000:1.0:1789760697.018017:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707278020797 time 707317799016 state 0x2 ref 0 00000001:00080000:0.0:1789760697.020091:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972392 is committed 00000100:00100000:2.0:1789760697.021524:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760697.021650:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493628672 00000100:00100000:0.0:1789760697.021659:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493628672 00000100:00100000:0.0:1789760697.021723:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707278020797 time 707317799016 state 0x0 ref 1 00000100:00100000:0.0:1789760697.021737:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707318059432 time 707317799016 state 0x1 ref 1 00000100:00100000:0.0:1789760697.021764:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707318059432 time 707362800245 state 0x0 ref 1 00000100:00100000:0.0:1789760697.021776:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493628672:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760697.021834:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20783104, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:0.0:1789760697.021842:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e040e8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.021870:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760697.021875:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760697.021881:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e040e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.021894:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e040e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760697.022140:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760697.065586:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972393 is committed 00002000:00010000:0.0:1789760697.065670:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20783104 -> 20787200 00010000:00010000:0.0:1789760697.065686:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e040e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.065699:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e040e8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.065707:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e040e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.065715:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760697.065720:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e040e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.065730:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e040e8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.065748:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e040e8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.065757:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e040e8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.065947:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493628672:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44167us (44425us total) trans 4294972393 rc 0/0 00000100:00100000:0.0:1789760697.065996:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5104 00000100:00100000:0.0:1789760697.066008:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707318059432 time 707362800245 state 0x2 ref 0 00000100:00100000:2.0:1789760697.069419:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760697.069519:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493628800 00000100:00100000:1.0:1789760697.069528:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493628800 00000100:00100000:1.0:1789760697.069582:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707318059432 time 707362800245 state 0x0 ref 1 00000100:00100000:1.0:1789760697.069595:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707363060661 time 707362800245 state 0x1 ref 1 00000100:00100000:1.0:1789760697.069659:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707363060661 time 707410694557 state 0x0 ref 1 00000100:00100000:1.0:1789760697.069673:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493628800:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760697.069723:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20787200, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:1.0:1789760697.069731:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040ef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.069744:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760697.069748:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760697.069754:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.069766:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760697.103521:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972394 is committed 00002000:00010000:2.0:1789760697.103630:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20787200 -> 20791296 00010000:00010000:2.0:1789760697.103647:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.103660:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040ef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.103669:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.103676:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760697.103681:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.103690:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.103710:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040ef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.103718:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e040ef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760697.104003:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493628800:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34285us (34546us total) trans 4294972394 rc 0/0 00000100:00100000:2.0:1789760697.104015:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5105 00000100:00100000:2.0:1789760697.104025:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707363060661 time 707410694557 state 0x2 ref 0 00000100:00100000:2.0:1789760697.108244:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760697.108548:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493628928 00000100:00100000:1.0:1789760697.108557:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493628928 00000100:00100000:1.0:1789760697.108610:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707363060661 time 707410694557 state 0x0 ref 1 00000100:00100000:1.0:1789760697.108626:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707410954973 time 707410694557 state 0x1 ref 1 00000100:00100000:1.0:1789760697.108654:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707410954973 time 707449689288 state 0x0 ref 1 00000100:00100000:1.0:1789760697.108667:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493628928:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760697.108713:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20791296, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:1.0:1789760697.108723:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e040f6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.108738:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760697.108743:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760697.108749:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e040f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.108763:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e040f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760697.140530:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972395 is committed 00002000:00010000:1.0:1789760697.140599:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20791296 -> 20795392 00010000:00010000:1.0:1789760697.140613:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e040f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.140631:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e040f6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.140639:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e040f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.140647:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760697.140652:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e040f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.140661:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e040f6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.140680:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e040f6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.140688:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e040f6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760697.140784:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493628928:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32115us (32545us total) trans 4294972395 rc 0/0 00000100:00100000:1.0:1789760697.140795:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5106 00000100:00100000:1.0:1789760697.140804:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707410954973 time 707449689288 state 0x2 ref 0 00000100:00100000:3.0:1789760697.145276:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760697.147188:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493629056 00000100:00100000:2.0:1789760697.147201:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493629056 00000100:00100000:2.0:1789760697.147274:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707410954973 time 707449689288 state 0x0 ref 1 00000100:00100000:2.0:1789760697.147807:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707449949704 time 707449689288 state 0x1 ref 1 00000100:00100000:2.0:1789760697.147859:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707449949704 time 707488894287 state 0x0 ref 1 00000100:00100000:2.0:1789760697.148105:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493629056:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760697.148153:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20795392, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:2.0:1789760697.148162:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e040fd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.148178:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760697.148183:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760697.148189:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e040fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.148204:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e040fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760697.211143:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972396 is committed 00002000:00010000:2.0:1789760697.211460:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20795392 -> 20799488 00010000:00010000:2.0:1789760697.211475:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e040fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.211489:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e040fd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.211498:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e040fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.211506:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760697.211511:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e040fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.211521:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e040fd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.211541:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e040fd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.211550:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e040fd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760697.211852:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493629056:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63748us (66579us total) trans 4294972396 rc 0/0 00000100:00100000:2.0:1789760697.211878:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5107 00000100:00100000:2.0:1789760697.211887:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707449949704 time 707488894287 state 0x2 ref 0 00000100:00100000:2.0:1789760697.212202:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc5834e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3303:1876699519244160:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760697.212256:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760697.212436:0:6691:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519244160 00000100:00100000:3.0:1789760697.212444:0:6691:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519244160 00000100:00100000:3.0:1789760697.212498:0:6691:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 182 00000100:00100000:3.0:1789760697.212508:0:6691:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d0a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876699519244160:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789760697.212528:0:6691:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526319616 pending 0 free 3837788160 avail 3823108096 00000100:00100000:3.0:1789760697.212568:0:6691:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d0a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876699519244160:12345-0@lo:13:osp-pre-0-0.0 Request processed in 62us (311us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760697.212576:0:6691:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 182 00000100:00100000:3.0:1789760697.212613:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc5834e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3303:1876699519244160:0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1789760697.212668:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760697.217388:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760697.217492:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493629184 00000100:00100000:0.0:1789760697.217501:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493629184 00000100:00100000:0.0:1789760697.217554:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707449949704 time 707488894287 state 0x0 ref 1 00000100:00100000:0.0:1789760697.217570:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707489154703 time 707488894287 state 0x1 ref 1 00000100:00100000:0.0:1789760697.217690:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707489154703 time 707558724619 state 0x0 ref 1 00000100:00100000:0.0:1789760697.217704:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493629184:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760697.217749:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20799488, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:0.0:1789760697.217758:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04104 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.217773:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760697.217778:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760697.217784:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04104 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.217804:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04104 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760697.260636:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972397 is committed 00002000:00010000:3.0:1789760697.260758:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20799488 -> 20803584 00010000:00010000:3.0:1789760697.260774:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04104 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.260793:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04104 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.260803:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04104 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.260820:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760697.260824:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04104 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.260833:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04104 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.260852:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04104 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.260872:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04104 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760697.260994:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493629184:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43256us (43579us total) trans 4294972397 rc 0/0 00000100:00100000:3.0:1789760697.261005:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5108 00000100:00100000:3.0:1789760697.261015:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707489154703 time 707558724619 state 0x2 ref 0 00000100:00100000:0.0:1789760697.264090:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760697.264403:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493629312 00000100:00100000:1.0:1789760697.264414:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493629312 00000100:00100000:1.0:1789760697.264474:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707489154703 time 707558724619 state 0x0 ref 1 00000100:00100000:1.0:1789760697.264493:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707558985035 time 707558724619 state 0x1 ref 1 00000100:00100000:1.0:1789760697.264530:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707558985035 time 707605564175 state 0x0 ref 1 00000100:00100000:1.0:1789760697.264544:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493629312:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760697.264600:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20803584, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:1.0:1789760697.264609:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0410b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.264625:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760697.264629:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760697.264635:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0410b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.264649:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0410b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760697.296941:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972398 is committed 00002000:00010000:2.0:1789760697.297446:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20803584 -> 20807680 00010000:00010000:2.0:1789760697.297460:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0410b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.297473:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0410b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.297482:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0410b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.297490:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760697.297495:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0410b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.297505:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0410b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.297525:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0410b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.297533:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0410b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760697.297748:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493629312:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33200us (33661us total) trans 4294972398 rc 0/0 00000100:00100000:2.0:1789760697.297760:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5109 00000100:00100000:2.0:1789760697.297772:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707558985035 time 707605564175 state 0x2 ref 0 00000100:00100000:2.0:1789760697.301795:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760697.301888:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493629440 00000100:00100000:0.0:1789760697.301899:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493629440 00000100:00100000:0.0:1789760697.301957:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707558985035 time 707605564175 state 0x0 ref 1 00000100:00100000:0.0:1789760697.302002:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707605824591 time 707605564175 state 0x1 ref 1 00000100:00100000:0.0:1789760697.302030:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707605824591 time 707643065417 state 0x0 ref 1 00000100:00100000:0.0:1789760697.302042:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493629440:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760697.302091:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20807680, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:0.0:1789760697.302100:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04112 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.302116:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760697.302121:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760697.302127:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04112 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.302141:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04112 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760697.354481:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972399 is committed 00002000:00010000:2.0:1789760697.354581:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20807680 -> 20811776 00010000:00010000:2.0:1789760697.354598:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04112 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.354611:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04112 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.354619:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04112 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.354626:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760697.354631:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04112 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.354640:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04112 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.354664:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04112 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.354672:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04112 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760697.354762:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493629440:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52717us (52972us total) trans 4294972399 rc 0/0 00000100:00100000:2.0:1789760697.354773:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5110 00000100:00100000:2.0:1789760697.354782:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707605824591 time 707643065417 state 0x2 ref 0 00000100:00100000:2.0:1789760697.358760:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760697.358925:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493629568 00000100:00100000:0.0:1789760697.358935:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493629568 00000100:00100000:0.0:1789760697.359120:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707605824591 time 707643065417 state 0x0 ref 1 00000100:00100000:0.0:1789760697.359135:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707643325833 time 707643065417 state 0x1 ref 1 00000100:00100000:0.0:1789760697.359156:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707643325833 time 707700191960 state 0x0 ref 1 00000100:00100000:0.0:1789760697.359169:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493629568:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760697.359217:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20811776, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:0.0:1789760697.359226:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04119 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.359242:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760697.359247:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760697.359253:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04119 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.359266:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04119 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760697.470375:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972400 is committed 00002000:00010000:2.0:1789760697.470615:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20811776 -> 20815872 00010000:00010000:2.0:1789760697.470631:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04119 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.470644:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04119 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.470653:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04119 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.470660:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760697.470664:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04119 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.470679:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04119 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.470698:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04119 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.470705:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04119 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760697.470800:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493629568:12345-192.168.203.60@tcp:4:dd.0 Request processed in 111627us (112042us total) trans 4294972400 rc 0/0 00000100:00100000:2.0:1789760697.470819:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5111 00000100:00100000:2.0:1789760697.470830:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707643325833 time 707700191960 state 0x2 ref 0 00000100:00100000:1.0:1789760697.480052:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760697.480296:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493629696 00000100:00100000:3.0:1789760697.480306:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493629696 00000100:00100000:3.0:1789760697.480358:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707643325833 time 707700191960 state 0x0 ref 1 00000100:00100000:3.0:1789760697.480374:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707700452376 time 707700191960 state 0x1 ref 1 00000100:00100000:3.0:1789760697.480404:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707700452376 time 707821439363 state 0x0 ref 1 00000100:00100000:3.0:1789760697.480416:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8556300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493629696:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760697.480474:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20815872, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:3.0:1789760697.480491:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04120 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.480507:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760697.480512:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760697.480518:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04120 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.480531:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04120 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760697.549884:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20815872 -> 20819968 00010000:00010000:1.0:1789760697.549905:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04120 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.549929:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04120 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.549938:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04120 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.549946:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760697.549950:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04120 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.549994:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04120 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.550014:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04120 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760697.550022:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04120 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760697.550121:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8556300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493629696:12345-192.168.203.60@tcp:4:dd.0 Request processed in 69702us (70072us total) trans 4294972401 rc 0/0 00000100:00100000:1.0:1789760697.550131:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5112 00000100:00100000:1.0:1789760697.550141:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707700452376 time 707821439363 state 0x2 ref 0 00000001:00080000:1.0:1789760697.550179:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972401 is committed 00000100:00100000:1.0:1789760697.553058:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760697.553141:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493629824 00000100:00100000:3.0:1789760697.553150:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493629824 00000100:00100000:3.0:1789760697.553213:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707700452376 time 707821439363 state 0x0 ref 1 00000100:00100000:3.0:1789760697.553227:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707821699779 time 707821439363 state 0x1 ref 1 00000100:00100000:3.0:1789760697.553250:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707821699779 time 707894286649 state 0x0 ref 1 00000100:00100000:3.0:1789760697.553264:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8555500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493629824:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760697.553308:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20819968, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:3.0:1789760697.553317:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04127 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.553331:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760697.553335:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760697.553341:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04127 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.553355:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04127 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760697.591956:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972402 is committed 00002000:00010000:2.0:1789760697.592759:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20819968 -> 20824064 00010000:00010000:2.0:1789760697.592776:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04127 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.592789:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04127 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.592798:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04127 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.592814:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760697.592820:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04127 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.592830:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04127 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.592849:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04127 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760697.592907:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04127 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760697.595485:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8555500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493629824:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42208us (42419us total) trans 4294972402 rc 0/0 00000100:00100000:3.0:1789760697.595499:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5113 00000100:00100000:3.0:1789760697.595508:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707821699779 time 707894286649 state 0x2 ref 0 00000100:00100000:2.0:1789760697.601890:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760697.602030:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493629952 00000100:00100000:0.0:1789760697.602041:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493629952 00000100:00100000:0.0:1789760697.602094:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707821699779 time 707894286649 state 0x0 ref 1 00000100:00100000:0.0:1789760697.602110:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707894547065 time 707894286649 state 0x1 ref 1 00000100:00100000:0.0:1789760697.602146:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707894547065 time 707943182112 state 0x0 ref 1 00000100:00100000:0.0:1789760697.602160:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493629952:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760697.602220:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20824064, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0xa007 00010000:00010000:0.0:1789760697.602229:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0412e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.602245:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760697.602250:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760697.602257:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0412e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.602271:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0412e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760697.661010:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972403 is committed 00002000:00010000:3.0:1789760697.661262:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20824064 -> 20828160 00002000:00010000:3.0:1789760697.661267:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760696 -> 1789760697 00002000:00010000:3.0:1789760697.661269:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760696 -> 1789760697 00010000:00010000:3.0:1789760697.661282:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0412e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.661295:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0412e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.661304:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0412e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.661311:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760697.661316:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0412e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.661326:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0412e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.661347:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0412e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.661355:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0412e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760697.661439:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493629952:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59277us (59554us total) trans 4294972403 rc 0/0 00000100:00100000:3.0:1789760697.661821:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5114 00000100:00100000:3.0:1789760697.661831:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707894547065 time 707943182112 state 0x2 ref 0 00000100:00100000:1.0:1789760697.672763:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760697.672872:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493630080 00000100:00100000:0.0:1789760697.672881:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493630080 00000100:00100000:0.0:1789760697.672934:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707894547065 time 707943182112 state 0x0 ref 1 00000100:00100000:0.0:1789760697.672949:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707943442528 time 707943182112 state 0x1 ref 1 00000100:00100000:0.0:1789760697.673011:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707943442528 time 708014046483 state 0x0 ref 1 00000100:00100000:0.0:1789760697.673024:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493630080:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760697.673076:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20828160, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:0.0:1789760697.673085:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04135 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.673100:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760697.673105:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760697.673111:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04135 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.673126:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04135 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760697.716106:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972404 is committed 00002000:00010000:3.0:1789760697.716333:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20828160 -> 20832256 00010000:00010000:3.0:1789760697.716347:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04135 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.716365:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04135 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.716373:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04135 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.716381:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760697.716385:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04135 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.716394:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04135 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.716411:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04135 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.716419:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04135 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760697.716508:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493630080:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43482us (43750us total) trans 4294972404 rc 0/0 00000100:00100000:3.0:1789760697.716517:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5115 00000100:00100000:3.0:1789760697.716527:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707943442528 time 708014046483 state 0x2 ref 0 00000100:00100000:1.0:1789760697.721948:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760697.722177:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493630208 00000100:00100000:3.0:1789760697.722188:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493630208 00000100:00100000:3.0:1789760697.722239:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 707943442528 time 708014046483 state 0x0 ref 1 00000100:00100000:3.0:1789760697.722256:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708014306899 time 708014046483 state 0x1 ref 1 00000100:00100000:3.0:1789760697.722295:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708014306899 time 708063322606 state 0x0 ref 1 00000100:00100000:3.0:1789760697.722309:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8557b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493630208:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760697.722354:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20832256, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:3.0:1789760697.722363:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0413c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.722380:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760697.722385:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760697.722391:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0413c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.722406:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0413c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760697.723680:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620c8554380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519244544:0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.723729:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760697.723782:0:3306:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1789760697.723789:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620c8556a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3306:1876699519244416:0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.723827:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760697.723843:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4d89180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519244928:0@lo:400:kworker.0 00000100:00100000:0.0:1789760697.723943:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620c8555180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3304:1876699519244288:0@lo:400:kworker.0 00000100:00100000:2.0:1789760697.724046:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4d88700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3303:1876699519244672:0@lo:400:kworker.0 00000100:00100000:0.0:1789760697.725274:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760697.725411:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4d89880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3304:1876699519244800:0@lo:400:kworker.0 00000100:00100000:0.0:1789760697.725439:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760697.725497:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519244416 00000100:00100000:0.0:1789760697.725504:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519244416 00000100:00100000:0.0:1789760697.725549:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519244800 00000100:00100000:0.0:1789760697.725553:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519244800 00000100:00100000:0.0:1789760697.725584:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 439 00000100:00100000:0.0:1789760697.725593:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d89500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3306:x1876699519244416:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760697.725640:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d89500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3306:x1876699519244416:12345-0@lo:400:kworker.0 Request processed in 47us (1811us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760697.725647:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 439 00000100:00100000:0.0:1789760697.725664:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 440 00000100:00100000:0.0:1789760697.725670:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3304:x1876699519244800:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760697.725700:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3304:x1876699519244800:12345-0@lo:400:kworker.0 Request processed in 30us (260us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760697.725706:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 440 00000100:00080000:0.0:1789760697.725726:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760697.725756:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4d89880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3304:1876699519244800:0@lo:400:kworker.0 00000100:00100000:2.0:1789760697.728298:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760697.728488:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760697.728526:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519244672 00000100:00100000:1.0:1789760697.728534:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519244672 00000100:00100000:1.0:1789760697.728585:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 441 00000100:00100000:1.0:1789760697.728593:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4361880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876699519244672:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.728642:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4361880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876699519244672:12345-0@lo:400:kworker.0 Request processed in 51us (349us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760697.728648:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 441 00000100:00100000:1.0:1789760697.728670:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519244544 00000100:00100000:1.0:1789760697.728673:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519244544 00000100:00100000:1.0:1789760697.728693:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519244928 00000100:00100000:1.0:1789760697.728696:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519244928 00000100:00100000:1.0:1789760697.728715:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 160 00000100:00100000:1.0:1789760697.728721:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519244544:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.728753:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519244544:12345-0@lo:400:kworker.0 Request processed in 33us (5024us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760697.728759:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 160 00000100:00100000:1.0:1789760697.728772:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 161 00000100:00100000:1.0:1789760697.728778:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d89c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519244928:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.728815:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d89c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519244928:12345-0@lo:400:kworker.0 Request processed in 26us (316us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760697.728821:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 161 00000100:00100000:1.0:1789760697.728871:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4d88700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3303:1876699519244672:0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.728902:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4d89180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519244928:0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.728913:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620c8554380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519244544:0@lo:400:kworker.0 00000100:00100000:1.0:1789760697.728931:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620c8556a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3306:1876699519244416:0@lo:400:kworker.0 00000100:00080000:1.0:1789760697.730679:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789760697.730687:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760697.730889:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519244288 00000100:00100000:3.0:1789760697.730897:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519244288 00000100:00100000:3.0:1789760697.731011:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 382 00000100:00100000:3.0:1789760697.731021:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7bff850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3304:x1876699519244288:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760697.731102:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7bff850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3304:x1876699519244288:12345-0@lo:400:kworker.0 Request processed in 83us (5829us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760697.731110:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 382 00000100:00100000:1.0:1789760697.731183:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620c8555180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3304:1876699519244288:0@lo:400:kworker.0 00000001:00080000:0.0:1789760697.796258:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972405 is committed 00002000:00010000:3.0:1789760697.796493:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20832256 -> 20836352 00010000:00010000:3.0:1789760697.796509:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0413c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.796522:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0413c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.796532:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0413c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.796540:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760697.796544:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0413c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.796554:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0413c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.796572:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0413c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.796580:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0413c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760697.796673:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8557b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493630208:12345-192.168.203.60@tcp:4:dd.0 Request processed in 74362us (74729us total) trans 4294972405 rc 0/0 00000100:00100000:3.0:1789760697.796682:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5116 00000100:00100000:3.0:1789760697.796693:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708014306899 time 708063322606 state 0x2 ref 0 00000100:00100000:2.0:1789760697.799816:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760697.800029:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493630336 00000100:00100000:3.0:1789760697.800039:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493630336 00000100:00100000:3.0:1789760697.800093:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708014306899 time 708063322606 state 0x0 ref 1 00000100:00100000:3.0:1789760697.800108:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708063583022 time 708063322606 state 0x1 ref 1 00000100:00100000:3.0:1789760697.800130:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708063583022 time 708141165794 state 0x0 ref 1 00000100:00100000:3.0:1789760697.800143:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493630336:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760697.800189:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20836352, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:3.0:1789760697.800210:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04143 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.800225:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760697.800230:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760697.800237:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04143 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.800250:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04143 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760697.854738:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972406 is committed 00002000:00010000:3.0:1789760697.854834:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20836352 -> 20840448 00010000:00010000:3.0:1789760697.854848:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04143 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.854913:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04143 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.854923:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04143 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.854930:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760697.854935:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04143 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.854943:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04143 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.854995:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04143 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.855004:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04143 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760697.855083:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493630336:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54939us (55286us total) trans 4294972406 rc 0/0 00000100:00100000:3.0:1789760697.855093:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5117 00000100:00100000:3.0:1789760697.855103:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708063583022 time 708141165794 state 0x2 ref 0 00010000:00010000:1.0:1789760697.858442:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760697.859031:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760697.859127:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493630464 00000100:00100000:3.0:1789760697.859138:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493630464 00000100:00100000:3.0:1789760697.859206:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708063583022 time 708141165794 state 0x0 ref 1 00000100:00100000:3.0:1789760697.859225:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708141426210 time 708141165794 state 0x1 ref 1 00000100:00100000:3.0:1789760697.859311:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708141426210 time 708200344914 state 0x0 ref 1 00000100:00100000:3.0:1789760697.859325:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493630464:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760697.859384:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20840448, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:3.0:1789760697.859393:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0414a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.859410:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760697.859415:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760697.859421:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0414a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760697.859436:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0414a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760697.898150:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20840448 -> 20844544 00010000:00010000:0.0:1789760697.898168:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0414a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.898183:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0414a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.898192:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0414a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.898199:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760697.898204:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0414a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.898214:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0414a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.898233:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0414a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.898241:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0414a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.898333:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493630464:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39006us (39367us total) trans 4294972407 rc 0/0 00000100:00100000:0.0:1789760697.898342:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5118 00000100:00100000:0.0:1789760697.898352:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708141426210 time 708200344914 state 0x2 ref 0 00000001:00080000:2.0:1789760697.898567:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972407 is committed 00000100:00100000:2.0:1789760697.902773:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760697.902849:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493630592 00000100:00100000:0.0:1789760697.902865:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493630592 00000100:00100000:0.0:1789760697.902920:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708141426210 time 708200344914 state 0x0 ref 1 00000100:00100000:0.0:1789760697.902934:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708200605330 time 708200344914 state 0x1 ref 1 00000100:00100000:0.0:1789760697.902981:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708200605330 time 708243994646 state 0x0 ref 1 00000100:00100000:0.0:1789760697.902994:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493630592:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760697.903035:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20844544, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:0.0:1789760697.903043:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04151 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.903058:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760697.903063:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760697.903069:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04151 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.903088:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04151 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760697.932676:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972408 is committed 00002000:00010000:0.0:1789760697.932845:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20844544 -> 20848640 00010000:00010000:0.0:1789760697.932871:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04151 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.932884:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04151 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.932893:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04151 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.932900:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760697.933028:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04151 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.933038:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04151 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.933057:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04151 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.933065:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04151 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.933159:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493630592:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30163us (30390us total) trans 4294972408 rc 0/0 00000100:00100000:0.0:1789760697.933170:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5119 00000100:00100000:0.0:1789760697.933180:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708200605330 time 708243994646 state 0x2 ref 0 00000100:00100000:3.0:1789760697.941406:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760697.946000:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493630720 00000100:00100000:1.0:1789760697.946090:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493630720 00000100:00100000:0.0:1789760697.947124:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708200605330 time 708243994646 state 0x0 ref 1 00000100:00100000:0.0:1789760697.947141:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708244255062 time 708243994646 state 0x1 ref 1 00000100:00100000:0.0:1789760697.947180:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708244255062 time 708288215081 state 0x0 ref 1 00000100:00100000:0.0:1789760697.947193:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493630720:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760697.947241:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20848640, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:0.0:1789760697.947249:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04158 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.947264:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760697.947268:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760697.947274:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04158 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.947387:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04158 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760697.998211:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972409 is committed 00002000:00010000:0.0:1789760697.998317:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20848640 -> 20852736 00010000:00010000:0.0:1789760697.998333:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04158 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.998345:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04158 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.998354:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04158 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.998361:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760697.998366:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04158 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.998375:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04158 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.998393:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04158 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.998402:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04158 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.998493:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493630720:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51299us (57093us total) trans 4294972409 rc 0/0 00000100:00100000:0.0:1789760697.998505:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5120 00000100:00100000:0.0:1789760697.998515:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708244255062 time 708288215081 state 0x2 ref 0 00000100:00100000:1.0:1789760698.004447:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760698.004562:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493630848 00000100:00100000:3.0:1789760698.004573:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493630848 00000100:00100000:3.0:1789760698.004631:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708244255062 time 708288215081 state 0x0 ref 1 00000100:00100000:3.0:1789760698.004647:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708288475497 time 708288215081 state 0x1 ref 1 00000100:00100000:3.0:1789760698.004707:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708288475497 time 708345742435 state 0x0 ref 1 00000100:00100000:3.0:1789760698.004724:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493630848:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760698.004778:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20852736, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:3.0:1789760698.004787:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0415f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.004813:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760698.004818:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760698.004824:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0415f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.004839:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0415f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760698.035635:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972410 is committed 00002000:00010000:3.0:1789760698.035730:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20852736 -> 20856832 00010000:00010000:3.0:1789760698.035746:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0415f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.035759:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0415f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.035768:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0415f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.035776:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760698.035781:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0415f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.035791:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0415f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.035821:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0415f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.035829:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0415f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760698.036054:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493630848:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31330us (31612us total) trans 4294972410 rc 0/0 00000100:00100000:3.0:1789760698.036065:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5121 00000100:00100000:3.0:1789760698.036074:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708288475497 time 708345742435 state 0x2 ref 0 00000100:00100000:2.0:1789760698.039603:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760698.039730:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493630976 00000100:00100000:3.0:1789760698.039746:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493630976 00000100:00100000:3.0:1789760698.039827:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708288475497 time 708345742435 state 0x0 ref 1 00000100:00100000:3.0:1789760698.039845:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708346002851 time 708345742435 state 0x1 ref 1 00000100:00100000:3.0:1789760698.039884:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708346002851 time 708380919714 state 0x0 ref 1 00000100:00100000:3.0:1789760698.039899:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493630976:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760698.040000:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20856832, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:3.0:1789760698.040011:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04166 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.040030:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760698.040036:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760698.040043:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04166 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.040061:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04166 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760698.095044:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972411 is committed 00002000:00010000:3.0:1789760698.095146:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20856832 -> 20860928 00010000:00010000:3.0:1789760698.095160:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04166 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.095172:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04166 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.095180:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04166 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.095187:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760698.095192:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04166 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.095201:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04166 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.095219:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04166 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.095227:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04166 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760698.095349:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493630976:12345-192.168.203.60@tcp:4:dd.0 Request processed in 55448us (55748us total) trans 4294972411 rc 0/0 00000100:00100000:3.0:1789760698.095359:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5122 00000100:00100000:3.0:1789760698.095367:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708346002851 time 708380919714 state 0x2 ref 0 00000100:00100000:2.0:1789760698.098235:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760698.102905:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493631104 00000100:00100000:0.0:1789760698.102920:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493631104 00000100:00100000:0.0:1789760698.102997:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708346002851 time 708380919714 state 0x0 ref 1 00000100:00100000:0.0:1789760698.103013:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708381180130 time 708380919714 state 0x1 ref 1 00000100:00100000:0.0:1789760698.103048:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708381180130 time 708444082955 state 0x0 ref 1 00000100:00100000:0.0:1789760698.103062:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493631104:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760698.103111:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20860928, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:0.0:1789760698.103120:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0416d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.103136:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760698.103140:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760698.103146:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0416d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.103161:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0416d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760698.142789:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972412 is committed 00002000:00010000:1.0:1789760698.143135:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20860928 -> 20865024 00010000:00010000:1.0:1789760698.143150:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0416d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.143162:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0416d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.143171:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0416d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.143184:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760698.143188:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0416d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.143196:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0416d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.143215:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0416d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.143223:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0416d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760698.143416:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493631104:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40351us (45182us total) trans 4294972412 rc 0/0 00000100:00100000:1.0:1789760698.143427:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5123 00000100:00100000:1.0:1789760698.143437:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708381180130 time 708444082955 state 0x2 ref 0 00000100:00100000:0.0:1789760698.145571:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760698.146628:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493631232 00000100:00100000:3.0:1789760698.146639:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493631232 00000100:00100000:0.0:1789760698.146835:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708381180130 time 708444082955 state 0x0 ref 1 00000100:00100000:0.0:1789760698.146850:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708444343371 time 708444082955 state 0x1 ref 1 00000100:00100000:0.0:1789760698.146997:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708444343371 time 708488031096 state 0x0 ref 1 00000100:00100000:0.0:1789760698.147010:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493631232:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760698.147058:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20865024, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:0.0:1789760698.147067:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04174 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.147083:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760698.147087:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760698.147094:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04174 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.147108:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04174 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760698.176621:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972413 is committed 00002000:00010000:3.0:1789760698.176785:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20865024 -> 20869120 00010000:00010000:3.0:1789760698.176810:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04174 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.176823:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04174 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.176832:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04174 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.176839:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760698.176844:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04174 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.176853:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04174 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.176883:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04174 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.176895:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04174 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760698.177070:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493631232:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30057us (31503us total) trans 4294972413 rc 0/0 00000100:00100000:3.0:1789760698.177081:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5124 00000100:00100000:3.0:1789760698.177091:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708444343371 time 708488031096 state 0x2 ref 0 00000100:00100000:2.0:1789760698.184264:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760698.184365:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493631360 00000100:00100000:1.0:1789760698.184375:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493631360 00000100:00100000:1.0:1789760698.184428:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708444343371 time 708488031096 state 0x0 ref 1 00000100:00100000:1.0:1789760698.184515:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708488291512 time 708488031096 state 0x1 ref 1 00000100:00100000:1.0:1789760698.184548:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708488291512 time 708525583791 state 0x0 ref 1 00000100:00100000:1.0:1789760698.184562:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493631360:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760698.184611:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20869120, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:1.0:1789760698.184621:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0417b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.184635:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760698.184639:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760698.184645:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0417b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.184657:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0417b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760698.234758:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620ffb89180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519245056:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760698.234840:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760698.235591:0:6691:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519245056 00000100:00100000:1.0:1789760698.235601:0:6691:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519245056 00000100:00100000:1.0:1789760698.235658:0:6691:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 183 00000100:00100000:1.0:1789760698.235667:0:6691:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519245056:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760698.235706:0:6691:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760698.235767:0:6691:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519245056:12345-0@lo:13:osp-pre-1-0.0 Request processed in 101us (927us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760698.235774:0:6691:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 183 00000100:00100000:3.0:1789760698.236039:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620ffb89180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519245056:0@lo:13:osp-pre-1-0.0 00002000:00010000:3.0:1789760698.247040:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20869120 -> 20873216 00010000:00010000:3.0:1789760698.247058:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0417b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.247070:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0417b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.247078:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0417b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.247086:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760698.247091:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0417b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.247105:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0417b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.247124:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0417b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.247131:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0417b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760698.247216:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493631360:12345-192.168.203.60@tcp:4:dd.0 Request processed in 62652us (63023us total) trans 4294972414 rc 0/0 00000100:00100000:3.0:1789760698.247226:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5125 00000100:00100000:3.0:1789760698.247237:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708488291512 time 708525583791 state 0x2 ref 0 00000001:00080000:2.0:1789760698.248012:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972414 is committed 00000100:00100000:1.0:1789760698.251200:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760698.251295:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493631488 00000100:00100000:3.0:1789760698.251304:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493631488 00000100:00100000:3.0:1789760698.251355:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708488291512 time 708525583791 state 0x0 ref 1 00000100:00100000:3.0:1789760698.251370:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708525844207 time 708525583791 state 0x1 ref 1 00000100:00100000:3.0:1789760698.251438:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708525844207 time 708592472456 state 0x0 ref 1 00000100:00100000:3.0:1789760698.251521:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493631488:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760698.251570:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20873216, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:3.0:1789760698.251579:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04182 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.251594:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760698.251599:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760698.251605:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04182 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.251628:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04182 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760698.284217:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972415 is committed 00002000:00010000:3.0:1789760698.284351:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20873216 -> 20877312 00010000:00010000:3.0:1789760698.284365:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04182 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.284377:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04182 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.284385:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04182 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.284392:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760698.284397:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04182 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.284407:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04182 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.284427:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04182 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.284435:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04182 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760698.284535:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493631488:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33012us (33340us total) trans 4294972415 rc 0/0 00000100:00100000:3.0:1789760698.284545:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5126 00000100:00100000:3.0:1789760698.284555:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708525844207 time 708592472456 state 0x2 ref 0 00000100:00100000:1.0:1789760698.291172:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760698.291250:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493631616 00000100:00100000:3.0:1789760698.291259:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493631616 00000100:00100000:3.0:1789760698.291309:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708525844207 time 708592472456 state 0x0 ref 1 00000100:00100000:3.0:1789760698.291324:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708592732872 time 708592472456 state 0x1 ref 1 00000100:00100000:3.0:1789760698.291354:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708592732872 time 708632389983 state 0x0 ref 1 00000100:00100000:3.0:1789760698.291367:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d89880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493631616:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760698.291413:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20877312, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:3.0:1789760698.291422:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04189 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.291437:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760698.291442:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760698.291448:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04189 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.291466:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04189 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760698.333562:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20877312 -> 20881408 00010000:00010000:1.0:1789760698.333583:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04189 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.333598:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04189 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.333607:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04189 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.333622:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760698.333627:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04189 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.333637:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04189 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.333658:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04189 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.333666:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04189 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760698.333762:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d89880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493631616:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42391us (42593us total) trans 4294972416 rc 0/0 00000100:00100000:1.0:1789760698.333772:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5127 00000100:00100000:1.0:1789760698.333782:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708592732872 time 708632389983 state 0x2 ref 0 00000001:00080000:2.0:1789760698.334351:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972416 is committed 00000100:00100000:1.0:1789760698.336488:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760698.336572:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493631744 00000100:00100000:0.0:1789760698.336582:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493631744 00000100:00100000:0.0:1789760698.336640:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708592732872 time 708632389983 state 0x0 ref 1 00000100:00100000:0.0:1789760698.336655:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708632650399 time 708632389983 state 0x1 ref 1 00000100:00100000:0.0:1789760698.336683:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708632650399 time 708677718708 state 0x0 ref 1 00000100:00100000:0.0:1789760698.336695:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8555180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493631744:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760698.336739:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20881408, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:0.0:1789760698.336749:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04190 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.336763:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760698.336768:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760698.336774:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04190 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.336789:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04190 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760698.372460:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20881408 -> 20885504 00010000:00010000:0.0:1789760698.372478:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04190 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.372490:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04190 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.372499:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04190 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.372507:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760698.372511:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04190 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.372520:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04190 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.372545:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04190 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.372553:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04190 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.372654:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8555180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493631744:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35956us (36169us total) trans 4294972417 rc 0/0 00000100:00100000:0.0:1789760698.372665:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5128 00000100:00100000:0.0:1789760698.372676:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708632650399 time 708677718708 state 0x2 ref 0 00000001:00080000:2.0:1789760698.373319:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972417 is committed 00000100:00100000:3.0:1789760698.376914:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760698.377173:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493631872 00000100:00100000:1.0:1789760698.377184:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493631872 00000100:00100000:1.0:1789760698.377239:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708632650399 time 708677718708 state 0x0 ref 1 00000100:00100000:1.0:1789760698.377254:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708677979124 time 708677718708 state 0x1 ref 1 00000100:00100000:1.0:1789760698.377293:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708677979124 time 708718327951 state 0x0 ref 1 00000100:00100000:1.0:1789760698.377305:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493631872:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760698.377352:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20885504, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:1.0:1789760698.377360:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04197 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.377377:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760698.377381:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760698.377387:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04197 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.377400:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04197 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760698.399789:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20885504 -> 20889600 00010000:00010000:0.0:1789760698.399816:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04197 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.399828:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04197 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.399838:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04197 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.399844:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760698.399849:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04197 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.399873:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04197 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.399894:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04197 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.399901:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04197 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.400035:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493631872:12345-192.168.203.60@tcp:4:dd.0 Request processed in 22725us (23124us total) trans 4294972418 rc 0/0 00000100:00100000:0.0:1789760698.400045:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5129 00000100:00100000:0.0:1789760698.400055:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708677979124 time 708718327951 state 0x2 ref 0 00000001:00080000:3.0:1789760698.400547:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972418 is committed 00000100:00100000:3.0:1789760698.408007:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760698.408056:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493632000 00000100:00100000:0.0:1789760698.408065:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493632000 00000100:00100000:0.0:1789760698.408119:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708677979124 time 708718327951 state 0x0 ref 1 00000100:00100000:0.0:1789760698.408134:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708718588367 time 708718327951 state 0x1 ref 1 00000100:00100000:0.0:1789760698.408176:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708718588367 time 708749205119 state 0x0 ref 1 00000100:00100000:0.0:1789760698.408188:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493632000:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760698.408235:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20889600, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:0.0:1789760698.408243:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0419e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.408259:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760698.408264:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760698.408269:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0419e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.408290:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0419e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760698.460532:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972419 is committed 00002000:00010000:0.0:1789760698.460864:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20889600 -> 20893696 00010000:00010000:0.0:1789760698.460880:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0419e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.460892:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0419e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.460901:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0419e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.460907:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760698.460912:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0419e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.460919:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0419e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.460938:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0419e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.460945:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0419e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.461067:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493632000:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52876us (53104us total) trans 4294972419 rc 0/0 00000100:00100000:0.0:1789760698.461077:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5130 00000100:00100000:0.0:1789760698.461086:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708718588367 time 708749205119 state 0x2 ref 0 00000100:00100000:0.0:1789760698.463737:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760698.464005:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493632128 00000100:00100000:3.0:1789760698.464015:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493632128 00000100:00100000:3.0:1789760698.464066:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708718588367 time 708749205119 state 0x0 ref 1 00000100:00100000:3.0:1789760698.464080:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708749465535 time 708749205119 state 0x1 ref 1 00000100:00100000:3.0:1789760698.464105:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708749465535 time 708805140443 state 0x0 ref 1 00000100:00100000:3.0:1789760698.464117:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493632128:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760698.464163:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20893696, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:3.0:1789760698.464171:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e041a5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.464186:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760698.464191:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760698.464197:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e041a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.464211:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e041a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760698.514018:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972420 is committed 00002000:00010000:3.0:1789760698.514190:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20893696 -> 20897792 00010000:00010000:3.0:1789760698.514206:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e041a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.514218:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e041a5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.514226:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e041a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.514234:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760698.514239:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e041a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.514249:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e041a5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.514268:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e041a5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.514276:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e041a5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760698.514366:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493632128:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50246us (50633us total) trans 4294972420 rc 0/0 00000100:00100000:3.0:1789760698.514380:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5131 00000100:00100000:3.0:1789760698.514389:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708749465535 time 708805140443 state 0x2 ref 0 00000100:00100000:3.0:1789760698.517465:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760698.517568:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493632256 00000100:00100000:0.0:1789760698.517578:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493632256 00000100:00100000:0.0:1789760698.517631:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708749465535 time 708805140443 state 0x0 ref 1 00000100:00100000:0.0:1789760698.517646:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708805400859 time 708805140443 state 0x1 ref 1 00000100:00100000:0.0:1789760698.517683:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708805400859 time 708858718313 state 0x0 ref 1 00000100:00100000:0.0:1789760698.517696:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493632256:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760698.517742:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20897792, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:0.0:1789760698.517751:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e041ac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.517766:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760698.517770:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760698.517776:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e041ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.517796:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e041ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760698.569297:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972421 is committed 00002000:00010000:2.0:1789760698.569384:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20897792 -> 20901888 00010000:00010000:2.0:1789760698.569400:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e041ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.569413:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e041ac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.569422:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e041ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.569429:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760698.569434:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e041ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.569506:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e041ac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.569588:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e041ac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.569597:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e041ac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760698.569685:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493632256:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51986us (52224us total) trans 4294972421 rc 0/0 00000100:00100000:2.0:1789760698.569699:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5132 00000100:00100000:2.0:1789760698.569708:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708805400859 time 708858718313 state 0x2 ref 0 00000100:00100000:1.0:1789760698.574628:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760698.574728:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493632384 00000100:00100000:0.0:1789760698.574740:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493632384 00000100:00100000:0.0:1789760698.574809:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708805400859 time 708858718313 state 0x0 ref 1 00000100:00100000:0.0:1789760698.574826:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708858978729 time 708858718313 state 0x1 ref 1 00000100:00100000:0.0:1789760698.574868:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708858978729 time 708915903472 state 0x0 ref 1 00000100:00100000:0.0:1789760698.574881:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8556a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493632384:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760698.574936:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20901888, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0xa007 00010000:00010000:0.0:1789760698.574945:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e041b3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.575002:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760698.575007:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760698.575013:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e041b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.575027:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e041b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760698.621100:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972422 is committed 00002000:00010000:0.0:1789760698.621326:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20901888 -> 20905984 00002000:00010000:0.0:1789760698.621332:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760697 -> 1789760698 00002000:00010000:0.0:1789760698.621335:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760697 -> 1789760698 00010000:00010000:0.0:1789760698.621347:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e041b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.621359:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e041b3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.621367:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e041b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.621374:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760698.621379:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e041b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.621387:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e041b3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.621403:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e041b3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.621410:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e041b3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.621510:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8556a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493632384:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46626us (46888us total) trans 4294972422 rc 0/0 00000100:00100000:0.0:1789760698.621533:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5133 00000100:00100000:0.0:1789760698.621542:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708858978729 time 708915903472 state 0x2 ref 0 00000100:00100000:0.0:1789760698.631332:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760698.631421:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493632512 00000100:00100000:2.0:1789760698.631430:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493632512 00000100:00100000:2.0:1789760698.631492:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708858978729 time 708915903472 state 0x0 ref 1 00000100:00100000:2.0:1789760698.631507:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708916163888 time 708915903472 state 0x1 ref 1 00000100:00100000:2.0:1789760698.631540:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708916163888 time 708972576470 state 0x0 ref 1 00000100:00100000:2.0:1789760698.631552:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493632512:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760698.631598:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20905984, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:2.0:1789760698.631608:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e041ba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.631624:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760698.631629:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760698.631636:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e041ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.631650:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e041ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760698.700678:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972423 is committed 00002000:00010000:2.0:1789760698.700825:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20905984 -> 20910080 00010000:00010000:2.0:1789760698.700841:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e041ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.700855:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e041ba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.700874:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e041ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.700881:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760698.700885:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e041ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.700893:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e041ba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.701048:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e041ba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.701056:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e041ba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760698.701138:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493632512:12345-192.168.203.60@tcp:4:dd.0 Request processed in 69583us (69810us total) trans 4294972423 rc 0/0 00000100:00100000:2.0:1789760698.701149:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5134 00000100:00100000:2.0:1789760698.701160:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708916163888 time 708972576470 state 0x2 ref 0 00000100:00100000:3.0:1789760698.706451:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760698.706754:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493632640 00000100:00100000:0.0:1789760698.706765:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493632640 00000100:00100000:0.0:1789760698.706832:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708916163888 time 708972576470 state 0x0 ref 1 00000100:00100000:0.0:1789760698.706847:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708972836886 time 708972576470 state 0x1 ref 1 00000100:00100000:0.0:1789760698.706891:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708972836886 time 709047926186 state 0x0 ref 1 00000100:00100000:0.0:1789760698.706914:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493632640:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760698.707000:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20910080, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:0.0:1789760698.707010:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e041c1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.707026:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760698.707031:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760698.707038:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e041c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.707052:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e041c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760698.763011:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972424 is committed 00002000:00010000:3.0:1789760698.763162:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20910080 -> 20914176 00010000:00010000:3.0:1789760698.763176:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e041c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.763186:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e041c1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.763192:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e041c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.763197:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760698.763200:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e041c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.763207:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e041c1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.763221:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e041c1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.763238:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e041c1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760698.763326:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493632640:12345-192.168.203.60@tcp:4:dd.0 Request processed in 56419us (56891us total) trans 4294972424 rc 0/0 00000100:00100000:3.0:1789760698.763336:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5135 00000100:00100000:3.0:1789760698.763345:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708972836886 time 709047926186 state 0x2 ref 0 00000100:00100000:1.0:1789760698.768067:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760698.768161:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493632768 00000100:00100000:0.0:1789760698.768172:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493632768 00000100:00100000:0.0:1789760698.768231:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 708972836886 time 709047926186 state 0x0 ref 1 00000100:00100000:0.0:1789760698.768250:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709048186602 time 709047926186 state 0x1 ref 1 00000100:00100000:0.0:1789760698.768369:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709048186602 time 709109403607 state 0x0 ref 1 00000100:00100000:0.0:1789760698.768381:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493632768:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760698.768429:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20914176, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:0.0:1789760698.768438:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e041c8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.768454:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760698.768458:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760698.768464:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e041c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.768478:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e041c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760698.795154:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972425 is committed 00002000:00010000:1.0:1789760698.795241:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20914176 -> 20918272 00010000:00010000:1.0:1789760698.795256:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e041c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.795271:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e041c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.795280:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e041c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.795288:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760698.795293:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e041c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.795302:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e041c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.795320:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e041c8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.795329:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e041c8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760698.795444:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493632768:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27060us (27381us total) trans 4294972425 rc 0/0 00000100:00100000:1.0:1789760698.795455:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5136 00000100:00100000:1.0:1789760698.795464:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709048186602 time 709109403607 state 0x2 ref 0 00000100:00100000:0.0:1789760698.798623:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760698.798739:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493632896 00000100:00100000:2.0:1789760698.798748:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493632896 00000100:00100000:2.0:1789760698.798805:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709048186602 time 709109403607 state 0x0 ref 1 00000100:00100000:2.0:1789760698.798821:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709109664023 time 709109403607 state 0x1 ref 1 00000100:00100000:2.0:1789760698.798893:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709109664023 time 709139927672 state 0x0 ref 1 00000100:00100000:2.0:1789760698.798906:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493632896:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760698.798956:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20918272, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:2.0:1789760698.798994:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e041cf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.799010:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760698.799015:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760698.799020:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e041cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.799035:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e041cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760698.839108:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972426 is committed 00002000:00010000:1.0:1789760698.839257:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20918272 -> 20922368 00010000:00010000:1.0:1789760698.839271:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e041cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.839284:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e041cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.839293:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e041cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.839300:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760698.839304:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e041cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.839313:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e041cf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.839332:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e041cf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760698.839340:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e041cf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760698.839435:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493632896:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40527us (40816us total) trans 4294972426 rc 0/0 00000100:00100000:1.0:1789760698.839511:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5137 00000100:00100000:1.0:1789760698.839522:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709109664023 time 709139927672 state 0x2 ref 0 00000100:00100000:3.0:1789760698.842709:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760698.843052:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493633024 00000100:00100000:2.0:1789760698.843062:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493633024 00000100:00100000:2.0:1789760698.843114:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709109664023 time 709139927672 state 0x0 ref 1 00000100:00100000:2.0:1789760698.843133:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709140188088 time 709139927672 state 0x1 ref 1 00000100:00100000:2.0:1789760698.843162:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709140188088 time 709184197649 state 0x0 ref 1 00000100:00100000:2.0:1789760698.843176:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493633024:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760698.843222:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20922368, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:2.0:1789760698.843231:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e041d6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.843245:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760698.843250:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760698.843256:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e041d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760698.843269:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e041d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.875188:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760698.885693:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972427 is committed 00002000:00010000:3.0:1789760698.885821:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20922368 -> 20926464 00010000:00010000:3.0:1789760698.885837:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e041d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.885994:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e041d6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.886006:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e041d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.886013:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760698.886017:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e041d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.886026:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e041d6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.886047:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e041d6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.886055:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e041d6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760698.886153:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493633024:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42975us (43448us total) trans 4294972427 rc 0/0 00000100:00100000:3.0:1789760698.886163:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5138 00000100:00100000:3.0:1789760698.886173:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709140188088 time 709184197649 state 0x2 ref 0 00000100:00100000:1.0:1789760698.891425:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760698.891585:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493633152 00000100:00100000:3.0:1789760698.891594:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493633152 00000100:00100000:3.0:1789760698.891650:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709140188088 time 709184197649 state 0x0 ref 1 00000100:00100000:3.0:1789760698.891666:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709184458065 time 709184197649 state 0x1 ref 1 00000100:00100000:3.0:1789760698.891769:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709184458065 time 709232803963 state 0x0 ref 1 00000100:00100000:3.0:1789760698.891782:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493633152:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760698.891864:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20926464, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:3.0:1789760698.891873:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e041dd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.891890:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760698.891896:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760698.891902:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e041dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760698.891916:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e041dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760699.013891:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972428 is committed 00002000:00010000:3.0:1789760699.015710:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20926464 -> 20930560 00010000:00010000:3.0:1789760699.015728:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e041dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.015742:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e041dd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.015752:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e041dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.015759:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760699.015764:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e041dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.015773:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e041dd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.015805:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e041dd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.015819:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e041dd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760699.017768:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493633152:12345-192.168.203.60@tcp:4:dd.0 Request processed in 125976us (126340us total) trans 4294972428 rc 0/0 00000100:00100000:2.0:1789760699.017781:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5139 00000100:00100000:2.0:1789760699.017791:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709184458065 time 709232803963 state 0x2 ref 0 00000100:00100000:1.0:1789760699.023307:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760699.023412:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493633280 00000100:00100000:2.0:1789760699.023422:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493633280 00000100:00100000:2.0:1789760699.023563:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709184458065 time 709232803963 state 0x0 ref 1 00000100:00100000:2.0:1789760699.023579:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709233064379 time 709232803963 state 0x1 ref 1 00000100:00100000:2.0:1789760699.023609:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709233064379 time 709364644544 state 0x0 ref 1 00000100:00100000:2.0:1789760699.023624:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493633280:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760699.023672:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20930560, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:2.0:1789760699.023681:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e041e4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.023698:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760699.023703:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760699.023710:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e041e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.023724:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e041e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760699.024950:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760699.113039:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972429 is committed 00002000:00010000:2.0:1789760699.113236:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20930560 -> 20934656 00010000:00010000:2.0:1789760699.113251:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e041e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.113265:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e041e4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.113351:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e041e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.113361:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760699.113366:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e041e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.113376:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e041e4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.113396:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e041e4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.113405:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e041e4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760699.113578:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493633280:12345-192.168.203.60@tcp:4:dd.0 Request processed in 89951us (90275us total) trans 4294972429 rc 0/0 00000100:00100000:2.0:1789760699.113589:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5140 00000100:00100000:2.0:1789760699.113599:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709233064379 time 709364644544 state 0x2 ref 0 00000100:00100000:3.0:1789760699.117790:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760699.117900:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493633408 00000100:00100000:1.0:1789760699.117910:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493633408 00000100:00100000:1.0:1789760699.117991:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709233064379 time 709364644544 state 0x0 ref 1 00000100:00100000:1.0:1789760699.118007:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709364904960 time 709364644544 state 0x1 ref 1 00000100:00100000:1.0:1789760699.118032:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709364904960 time 709459067717 state 0x0 ref 1 00000100:00100000:1.0:1789760699.118044:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493633408:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760699.118092:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20934656, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:1.0:1789760699.118100:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e041eb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.118117:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760699.118121:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760699.118128:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e041eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.118142:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e041eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760699.174207:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972430 is committed 00002000:00010000:1.0:1789760699.174532:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20934656 -> 20938752 00010000:00010000:1.0:1789760699.174549:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e041eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.174562:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e041eb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.174569:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e041eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.174576:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760699.174582:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e041eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.174590:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e041eb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.174611:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e041eb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.174618:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e041eb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760699.174758:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493633408:12345-192.168.203.60@tcp:4:dd.0 Request processed in 56711us (56972us total) trans 4294972430 rc 0/0 00000100:00100000:1.0:1789760699.174769:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5141 00000100:00100000:1.0:1789760699.174779:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709364904960 time 709459067717 state 0x2 ref 0 00000100:00100000:3.0:1789760699.182836:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760699.183062:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493633536 00000100:00100000:2.0:1789760699.183071:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493633536 00000100:00100000:2.0:1789760699.183123:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709364904960 time 709459067717 state 0x0 ref 1 00000100:00100000:2.0:1789760699.183138:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709459328133 time 709459067717 state 0x1 ref 1 00000100:00100000:2.0:1789760699.183222:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709459328133 time 709524255785 state 0x0 ref 1 00000100:00100000:2.0:1789760699.183236:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493633536:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760699.183291:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20938752, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:2.0:1789760699.183300:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e041f2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.183316:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760699.183320:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760699.183326:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e041f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.183341:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e041f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760699.234431:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20938752 -> 20942848 00010000:00010000:3.0:1789760699.234464:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e041f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.234479:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e041f2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.234489:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e041f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.234498:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760699.234503:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e041f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.234682:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e041f2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.234703:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e041f2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.234717:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e041f2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760699.234836:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493633536:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51596us (52003us total) trans 4294972431 rc 0/0 00000100:00100000:3.0:1789760699.234855:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5142 00000100:00100000:3.0:1789760699.234865:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709459328133 time 709524255785 state 0x2 ref 0 00000001:00080000:3.0:1789760699.234907:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972431 is committed 00000100:00100000:1.0:1789760699.239233:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760699.239380:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493633664 00000100:00100000:2.0:1789760699.239389:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493633664 00000100:00100000:2.0:1789760699.239440:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709459328133 time 709524255785 state 0x0 ref 1 00000100:00100000:2.0:1789760699.239462:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709524516201 time 709524255785 state 0x1 ref 1 00000100:00100000:2.0:1789760699.239487:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709524516201 time 709580523635 state 0x0 ref 1 00000100:00100000:2.0:1789760699.239500:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493633664:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760699.239705:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20942848, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:2.0:1789760699.239714:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e041f9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.239730:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760699.239734:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760699.239741:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e041f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.239760:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e041f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760699.302905:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20942848 -> 20946944 00010000:00010000:2.0:1789760699.302925:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e041f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.302938:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e041f9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.302947:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e041f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.302955:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760699.302979:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e041f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.302989:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e041f9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.303008:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e041f9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.303016:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e041f9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760699.303114:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493633664:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63611us (63886us total) trans 4294972432 rc 0/0 00000100:00100000:2.0:1789760699.303125:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5143 00000100:00100000:2.0:1789760699.303135:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709524516201 time 709580523635 state 0x2 ref 0 00000001:00080000:3.0:1789760699.303377:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972432 is committed 00000100:00100000:1.0:1789760699.306944:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760699.307120:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493633792 00000100:00100000:0.0:1789760699.307129:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493633792 00000100:00100000:0.0:1789760699.307250:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709524516201 time 709580523635 state 0x0 ref 1 00000100:00100000:0.0:1789760699.307266:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709580784051 time 709580523635 state 0x1 ref 1 00000100:00100000:0.0:1789760699.307303:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709580784051 time 709648338367 state 0x0 ref 1 00000100:00100000:0.0:1789760699.307318:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493633792:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760699.307525:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20946944, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:0.0:1789760699.307536:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04200 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.307553:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760699.307558:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760699.307564:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04200 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.307578:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04200 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760699.341099:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972433 is committed 00002000:00010000:1.0:1789760699.341261:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20946944 -> 20951040 00010000:00010000:1.0:1789760699.341341:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04200 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.341355:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04200 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.341364:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04200 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.341371:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760699.341376:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04200 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.341386:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04200 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.341407:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04200 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.341415:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04200 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760699.341564:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493633792:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34244us (34626us total) trans 4294972433 rc 0/0 00000100:00100000:1.0:1789760699.341576:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5144 00000100:00100000:1.0:1789760699.341585:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709580784051 time 709648338367 state 0x2 ref 0 00000100:00100000:1.0:1789760699.347235:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760699.347594:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493633920 00000100:00100000:2.0:1789760699.347604:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493633920 00000100:00100000:2.0:1789760699.347663:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709580784051 time 709648338367 state 0x0 ref 1 00000100:00100000:2.0:1789760699.347679:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709648598783 time 709648338367 state 0x1 ref 1 00000100:00100000:2.0:1789760699.347713:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709648598783 time 709688748848 state 0x0 ref 1 00000100:00100000:2.0:1789760699.347726:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493633920:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760699.347778:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20951040, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:2.0:1789760699.347788:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04207 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.347816:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760699.347820:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760699.347826:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04207 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.347840:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04207 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760699.384523:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972434 is committed 00002000:00010000:2.0:1789760699.384627:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20951040 -> 20955136 00010000:00010000:2.0:1789760699.384643:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04207 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.384657:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04207 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.384667:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04207 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.384675:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760699.384680:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04207 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.384689:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04207 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.384710:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04207 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760699.384717:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04207 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760699.384843:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493633920:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37114us (37614us total) trans 4294972434 rc 0/0 00000100:00100000:2.0:1789760699.384866:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5145 00000100:00100000:2.0:1789760699.384875:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709648598783 time 709688748848 state 0x2 ref 0 00000100:00100000:0.0:1789760699.390323:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760699.390510:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493634048 00000100:00100000:3.0:1789760699.390521:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493634048 00000100:00100000:3.0:1789760699.390575:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709648598783 time 709688748848 state 0x0 ref 1 00000100:00100000:3.0:1789760699.390591:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709689009264 time 709688748848 state 0x1 ref 1 00000100:00100000:3.0:1789760699.390619:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709689009264 time 709731655097 state 0x0 ref 1 00000100:00100000:3.0:1789760699.390631:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493634048:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760699.390678:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20955136, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:3.0:1789760699.390687:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0420e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.390709:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760699.390714:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760699.390719:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0420e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.390733:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0420e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760699.429729:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20955136 -> 20959232 00010000:00010000:3.0:1789760699.429747:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0420e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.429760:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0420e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.429769:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0420e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.429776:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760699.429782:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0420e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.429800:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0420e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.429820:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0420e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.429829:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0420e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760699.430078:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972435 is committed 00000100:00100000:3.0:1789760699.430456:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493634048:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39821us (40137us total) trans 4294972435 rc 0/0 00000100:00100000:3.0:1789760699.430467:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5146 00000100:00100000:3.0:1789760699.430477:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709689009264 time 709731655097 state 0x2 ref 0 00000100:00100000:3.0:1789760699.434231:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760699.434428:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493634176 00000100:00100000:0.0:1789760699.434446:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493634176 00000100:00100000:0.0:1789760699.434496:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709689009264 time 709731655097 state 0x0 ref 1 00000100:00100000:0.0:1789760699.434510:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709731915513 time 709731655097 state 0x1 ref 1 00000100:00100000:0.0:1789760699.434541:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709731915513 time 709775576782 state 0x0 ref 1 00000100:00100000:0.0:1789760699.434553:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493634176:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760699.434600:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20959232, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:0.0:1789760699.434609:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04215 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.434625:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760699.434630:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760699.434636:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04215 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.434651:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04215 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760699.481053:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972436 is committed 00002000:00010000:0.0:1789760699.481163:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20959232 -> 20963328 00010000:00010000:0.0:1789760699.481180:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04215 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.481193:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04215 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.481202:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04215 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.481210:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760699.481215:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04215 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.481224:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04215 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.481244:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04215 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.481252:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04215 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760699.481581:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493634176:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47024us (47353us total) trans 4294972436 rc 0/0 00000100:00100000:0.0:1789760699.481592:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5147 00000100:00100000:0.0:1789760699.481603:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709731915513 time 709775576782 state 0x2 ref 0 00000100:00100000:0.0:1789760699.485790:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760699.486088:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493634304 00000100:00100000:1.0:1789760699.486097:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493634304 00000100:00100000:1.0:1789760699.486148:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709731915513 time 709775576782 state 0x0 ref 1 00000100:00100000:1.0:1789760699.486162:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709775837198 time 709775576782 state 0x1 ref 1 00000100:00100000:1.0:1789760699.486189:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709775837198 time 709827224806 state 0x0 ref 1 00000100:00100000:1.0:1789760699.486201:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493634304:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760699.486253:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20963328, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:1.0:1789760699.486262:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0421c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.486288:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760699.486294:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760699.486302:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0421c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.486317:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0421c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760699.524815:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972437 is committed 00002000:00010000:1.0:1789760699.525048:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20963328 -> 20967424 00010000:00010000:1.0:1789760699.525065:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0421c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.525077:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0421c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.525085:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0421c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.525093:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760699.525098:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0421c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.525106:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0421c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.525125:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0421c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.525132:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0421c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760699.525386:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493634304:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39180us (39599us total) trans 4294972437 rc 0/0 00000100:00100000:1.0:1789760699.525397:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5148 00000100:00100000:1.0:1789760699.525407:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709775837198 time 709827224806 state 0x2 ref 0 00000100:00100000:3.0:1789760699.529749:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760699.530236:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493634432 00000100:00100000:1.0:1789760699.530246:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493634432 00000100:00100000:1.0:1789760699.530311:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709775837198 time 709827224806 state 0x0 ref 1 00000100:00100000:1.0:1789760699.530327:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709827485222 time 709827224806 state 0x1 ref 1 00000100:00100000:1.0:1789760699.530348:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709827485222 time 709871383187 state 0x0 ref 1 00000100:00100000:1.0:1789760699.530361:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493634432:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760699.530407:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20967424, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:1.0:1789760699.530416:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04223 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.530441:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760699.530447:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760699.530453:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04223 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.530466:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04223 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760699.590770:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972438 is committed 00002000:00010000:1.0:1789760699.590839:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20967424 -> 20971520 00010000:00010000:1.0:1789760699.590864:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04223 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.590883:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04223 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.590901:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04223 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.590909:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760699.590914:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04223 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.590923:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04223 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.590942:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04223 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.590950:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04223 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760699.591419:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493634432:12345-192.168.203.60@tcp:4:dd.0 Request processed in 61055us (61674us total) trans 4294972438 rc 0/0 00000100:00100000:1.0:1789760699.591441:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5149 00000100:00100000:1.0:1789760699.591451:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709827485222 time 709871383187 state 0x2 ref 0 00000100:00100000:1.0:1789760699.600581:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760699.600651:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493634560 00000100:00100000:3.0:1789760699.600662:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493634560 00000100:00100000:3.0:1789760699.600717:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709827485222 time 709871383187 state 0x0 ref 1 00000100:00100000:3.0:1789760699.600733:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709871643603 time 709871383187 state 0x1 ref 1 00000100:00100000:3.0:1789760699.600762:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709871643603 time 709941797684 state 0x0 ref 1 00000100:00100000:3.0:1789760699.600776:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493634560:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760699.600835:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20971520, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0xa007 00010000:00010000:3.0:1789760699.601154:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0422a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.601177:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760699.601182:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760699.601189:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0422a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.601203:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0422a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760699.669300:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20971520 -> 20975616 00002000:00010000:3.0:1789760699.669309:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760698 -> 1789760699 00002000:00010000:3.0:1789760699.669312:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760698 -> 1789760699 00002000:00010000:3.0:1789760699.669315:0:6692:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb blocks from disk: 40967 -> 43015 00010000:00010000:3.0:1789760699.669421:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0422a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.669491:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0422a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.669501:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0422a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.669509:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760699.669517:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0422a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.669527:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0422a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.669550:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0422a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.669559:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0422a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760699.669651:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493634560:12345-192.168.203.60@tcp:4:dd.0 Request processed in 68872us (69074us total) trans 4294972439 rc 0/0 00000100:00100000:3.0:1789760699.669661:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5150 00000100:00100000:3.0:1789760699.669670:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709871643603 time 709941797684 state 0x2 ref 0 00000001:00080000:1.0:1789760699.672402:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972439 is committed 00000100:00100000:0.0:1789760699.677649:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760699.678084:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493634688 00000100:00100000:2.0:1789760699.678093:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493634688 00000100:00100000:3.0:1789760699.678396:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709871643603 time 709941797684 state 0x0 ref 1 00000100:00100000:3.0:1789760699.678411:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709942058100 time 709941797684 state 0x1 ref 1 00000100:00100000:3.0:1789760699.678580:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709942058100 time 710019614251 state 0x0 ref 1 00000100:00100000:3.0:1789760699.678595:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493634688:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760699.678643:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20975616, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa807 00010000:00010000:3.0:1789760699.678652:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04231 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.678668:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760699.678673:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760699.678680:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04231 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.678693:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04231 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760699.726710:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972440 is committed 00002000:00010000:1.0:1789760699.728770:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20975616 -> 20979712 00010000:00010000:1.0:1789760699.728788:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04231 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.728812:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04231 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.728822:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04231 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.728829:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760699.728834:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04231 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.728957:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04231 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.729003:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04231 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.729012:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04231 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760699.729115:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493634688:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50518us (51469us total) trans 4294972440 rc 0/0 00000100:00100000:1.0:1789760699.729126:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5151 00000100:00100000:1.0:1789760699.729136:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709942058100 time 710019614251 state 0x2 ref 0 00000100:00100000:0.0:1789760699.732407:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760699.732558:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493634816 00000100:00100000:1.0:1789760699.732567:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493634816 00000100:00100000:1.0:1789760699.732626:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 709942058100 time 710019614251 state 0x0 ref 1 00000100:00100000:1.0:1789760699.732641:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710019874667 time 710019614251 state 0x1 ref 1 00000100:00100000:1.0:1789760699.732669:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710019874667 time 710073705259 state 0x0 ref 1 00000100:00100000:1.0:1789760699.732680:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493634816:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760699.732733:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20979712, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa807 00010000:00010000:1.0:1789760699.732743:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04238 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.732760:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760699.732764:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760699.732770:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04238 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.732785:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04238 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760699.772729:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972441 is committed 00002000:00010000:0.0:1789760699.772829:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20979712 -> 20983808 00010000:00010000:0.0:1789760699.772905:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04238 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.772919:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04238 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.772927:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04238 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.772935:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760699.772940:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04238 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.772954:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04238 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.773317:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04238 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.773326:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04238 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760699.773463:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493634816:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40779us (41063us total) trans 4294972441 rc 0/0 00000100:00100000:0.0:1789760699.773473:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5152 00000100:00100000:0.0:1789760699.773482:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710019874667 time 710073705259 state 0x2 ref 0 00000100:00100000:3.0:1789760699.777152:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760699.777286:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493634944 00000100:00100000:1.0:1789760699.777296:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493634944 00000100:00100000:1.0:1789760699.777352:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710019874667 time 710073705259 state 0x0 ref 1 00000100:00100000:1.0:1789760699.777371:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710073965675 time 710073705259 state 0x1 ref 1 00000100:00100000:1.0:1789760699.777404:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710073965675 time 710118439368 state 0x0 ref 1 00000100:00100000:1.0:1789760699.777416:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493634944:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760699.777466:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20983808, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa807 00010000:00010000:1.0:1789760699.777474:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0423f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.777490:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760699.777494:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760699.777500:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0423f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.777514:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0423f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760699.814180:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972442 is committed 00002000:00010000:1.0:1789760699.814525:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20983808 -> 20987904 00010000:00010000:1.0:1789760699.814542:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0423f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.814555:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0423f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.814564:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0423f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.814683:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760699.814689:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0423f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.814698:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0423f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.814717:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0423f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.814725:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0423f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760699.816095:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493634944:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38672us (38944us total) trans 4294972442 rc 0/0 00000100:00100000:1.0:1789760699.816107:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5153 00000100:00100000:1.0:1789760699.816117:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710073965675 time 710118439368 state 0x2 ref 0 00000100:00100000:1.0:1789760699.823249:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760699.823498:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493635072 00000100:00100000:3.0:1789760699.823508:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493635072 00000100:00100000:3.0:1789760699.823561:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710073965675 time 710118439368 state 0x0 ref 1 00000100:00100000:3.0:1789760699.823577:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710118699784 time 710118439368 state 0x1 ref 1 00000100:00100000:3.0:1789760699.823610:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710118699784 time 710164645166 state 0x0 ref 1 00000100:00100000:3.0:1789760699.823623:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493635072:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760699.823672:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20987904, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa807 00010000:00010000:3.0:1789760699.823682:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04246 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.823699:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760699.823704:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760699.823711:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04246 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.823726:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04246 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760699.878408:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972443 is committed 00002000:00010000:1.0:1789760699.878469:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20987904 -> 20992000 00010000:00010000:1.0:1789760699.878484:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04246 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.878497:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04246 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.878512:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04246 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.878520:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760699.878525:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04246 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.878534:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04246 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.878552:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04246 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.878560:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04246 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760699.878648:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493635072:12345-192.168.203.60@tcp:4:dd.0 Request processed in 55022us (55405us total) trans 4294972443 rc 0/0 00000100:00100000:1.0:1789760699.878658:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5154 00000100:00100000:1.0:1789760699.878667:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710118699784 time 710164645166 state 0x2 ref 0 00000100:00100000:1.0:1789760699.882832:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760699.882989:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493635200 00000100:00100000:3.0:1789760699.882999:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493635200 00000100:00100000:3.0:1789760699.883049:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710118699784 time 710164645166 state 0x0 ref 1 00000100:00100000:3.0:1789760699.883061:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710164905582 time 710164645166 state 0x1 ref 1 00000100:00100000:3.0:1789760699.883084:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710164905582 time 710224119797 state 0x0 ref 1 00000100:00100000:3.0:1789760699.883096:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493635200:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760699.883143:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20992000, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa807 00010000:00010000:3.0:1789760699.883151:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0424d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.883164:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760699.883167:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760699.883172:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0424d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.883284:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0424d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760699.903297:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760699.919693:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972444 is committed 00002000:00010000:1.0:1789760699.920679:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20992000 -> 20996096 00010000:00010000:1.0:1789760699.920696:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0424d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.920714:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0424d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.920723:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0424d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.920730:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760699.920734:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0424d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.920744:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0424d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.920762:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0424d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.920770:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0424d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760699.921058:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493635200:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37958us (38230us total) trans 4294972444 rc 0/0 00000100:00100000:1.0:1789760699.921068:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5155 00000100:00100000:1.0:1789760699.921077:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710164905582 time 710224119797 state 0x2 ref 0 00000100:00100000:3.0:1789760699.927586:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760699.927665:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493635328 00000100:00100000:1.0:1789760699.927674:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493635328 00000100:00100000:1.0:1789760699.927722:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710164905582 time 710224119797 state 0x0 ref 1 00000100:00100000:1.0:1789760699.927733:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710224380213 time 710224119797 state 0x1 ref 1 00000100:00100000:1.0:1789760699.927756:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710224380213 time 710268793175 state 0x0 ref 1 00000100:00100000:1.0:1789760699.927772:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9621016d0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493635328:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760699.927832:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20996096, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa807 00010000:00010000:1.0:1789760699.927841:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04254 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.927867:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760699.927871:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760699.927877:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04254 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.927898:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04254 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760699.953813:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 20996096 -> 21000192 00010000:00010000:1.0:1789760699.953834:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04254 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.953857:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04254 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.953873:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04254 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.953881:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760699.953886:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04254 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.954081:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04254 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.954102:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04254 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.954110:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04254 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760699.954220:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9621016d0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493635328:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26449us (26636us total) trans 4294972445 rc 0/0 00000100:00100000:1.0:1789760699.954230:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5156 00000100:00100000:1.0:1789760699.954240:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710224380213 time 710268793175 state 0x2 ref 0 00000001:00080000:3.0:1789760699.954733:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972445 is committed 00000100:00100000:2.0:1789760699.959624:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760699.959822:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493635456 00000100:00100000:1.0:1789760699.959833:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493635456 00000100:00100000:1.0:1789760699.960055:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710224380213 time 710268793175 state 0x0 ref 1 00000100:00100000:1.0:1789760699.960072:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710269053591 time 710268793175 state 0x1 ref 1 00000100:00100000:1.0:1789760699.960104:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710269053591 time 710301140251 state 0x0 ref 1 00000100:00100000:1.0:1789760699.960117:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493635456:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760699.960325:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21000192, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa807 00010000:00010000:1.0:1789760699.960334:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0425b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.960413:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760699.960418:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760699.960424:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0425b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760699.960504:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0425b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760700.000637:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21000192 -> 21004288 00010000:00010000:1.0:1789760700.000655:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0425b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.000668:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0425b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.000676:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0425b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.000684:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760700.000688:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0425b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.000697:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0425b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.000716:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0425b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.000724:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0425b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760700.000825:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493635456:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40705us (41207us total) trans 4294972446 rc 0/0 00000100:00100000:1.0:1789760700.000834:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5157 00000100:00100000:1.0:1789760700.000851:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710269053591 time 710301140251 state 0x2 ref 0 00000001:00080000:2.0:1789760700.001706:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972446 is committed 00000100:00100000:0.0:1789760700.008092:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760700.008299:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493635584 00000100:00100000:3.0:1789760700.008308:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493635584 00000100:00100000:3.0:1789760700.008469:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710269053591 time 710301140251 state 0x0 ref 1 00000100:00100000:3.0:1789760700.008485:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710301400667 time 710301140251 state 0x1 ref 1 00000100:00100000:3.0:1789760700.008515:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710301400667 time 710349550632 state 0x0 ref 1 00000100:00100000:3.0:1789760700.008528:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493635584:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760700.008574:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21004288, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa807 00010000:00010000:3.0:1789760700.008583:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04262 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.008598:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760700.008602:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760700.008608:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04262 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760700.008622:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04262 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760700.073634:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21004288 -> 21008384 00010000:00010000:1.0:1789760700.073654:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04262 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.073667:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04262 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.073678:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04262 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.073691:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760700.073696:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04262 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.073705:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04262 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.073725:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04262 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.073734:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04262 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760700.074012:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493635584:12345-192.168.203.60@tcp:4:dd.0 Request processed in 65363us (65807us total) trans 4294972447 rc 0/0 00000100:00100000:1.0:1789760700.074024:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5158 00000100:00100000:1.0:1789760700.074034:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710301400667 time 710349550632 state 0x2 ref 0 00000001:00080000:2.0:1789760700.075166:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972447 is committed 00000100:00100000:3.0:1789760700.088071:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760700.088347:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493635712 00000100:00100000:1.0:1789760700.088357:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493635712 00000100:00100000:1.0:1789760700.088416:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710301400667 time 710349550632 state 0x0 ref 1 00000100:00100000:1.0:1789760700.088431:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710349811048 time 710349550632 state 0x1 ref 1 00000100:00100000:1.0:1789760700.088510:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710349811048 time 710429545139 state 0x0 ref 1 00000100:00100000:1.0:1789760700.088527:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493635712:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760700.088576:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21008384, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa807 00010000:00010000:1.0:1789760700.088608:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04269 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.088624:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760700.088629:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760700.088636:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04269 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.088650:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04269 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760700.151011:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972448 is committed 00002000:00010000:2.0:1789760700.151208:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21008384 -> 21012480 00010000:00010000:2.0:1789760700.151225:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04269 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.151238:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04269 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.151247:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04269 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.151254:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760700.151260:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04269 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.151270:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04269 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.151296:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04269 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.151303:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04269 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760700.151398:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493635712:12345-192.168.203.60@tcp:4:dd.0 Request processed in 62868us (63332us total) trans 4294972448 rc 0/0 00000100:00100000:2.0:1789760700.151409:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5159 00000100:00100000:2.0:1789760700.151418:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710349811048 time 710429545139 state 0x2 ref 0 00000100:00100000:1.0:1789760700.161131:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760700.161235:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493635840 00000100:00100000:0.0:1789760700.161245:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493635840 00000100:00100000:0.0:1789760700.161299:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710349811048 time 710429545139 state 0x0 ref 1 00000100:00100000:0.0:1789760700.161314:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710429805555 time 710429545139 state 0x1 ref 1 00000100:00100000:0.0:1789760700.161344:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710429805555 time 710502379778 state 0x0 ref 1 00000100:00100000:0.0:1789760700.161359:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493635840:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760700.161403:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21012480, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa807 00010000:00010000:0.0:1789760700.161413:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04270 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.161438:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760700.161443:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760700.161449:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04270 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.161462:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04270 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760700.224746:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972449 is committed 00002000:00010000:0.0:1789760700.224835:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21012480 -> 21016576 00010000:00010000:0.0:1789760700.225024:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04270 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.225038:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04270 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.225046:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04270 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.225053:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760700.225058:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04270 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.225066:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04270 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.225090:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04270 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.225098:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04270 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760700.225348:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493635840:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63989us (64222us total) trans 4294972449 rc 0/0 00000100:00100000:2.0:1789760700.225359:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5160 00000100:00100000:2.0:1789760700.225368:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710429805555 time 710502379778 state 0x2 ref 0 00000100:00100000:1.0:1789760700.234171:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760700.234255:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493635968 00000100:00100000:2.0:1789760700.234275:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493635968 00000100:00100000:2.0:1789760700.234332:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710429805555 time 710502379778 state 0x0 ref 1 00000100:00100000:2.0:1789760700.234347:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710502640194 time 710502379778 state 0x1 ref 1 00000100:00100000:2.0:1789760700.234380:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710502640194 time 710575415070 state 0x0 ref 1 00000100:00100000:2.0:1789760700.234393:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493635968:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760700.234443:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21016576, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa807 00010000:00010000:2.0:1789760700.234454:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04277 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.234470:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760700.234474:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760700.234480:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04277 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.234494:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04277 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760700.312870:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972450 is committed 00002000:00010000:2.0:1789760700.313074:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21016576 -> 21020672 00010000:00010000:2.0:1789760700.313090:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04277 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.313102:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04277 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.313110:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04277 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.313117:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760700.313122:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04277 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.313131:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04277 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.313149:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04277 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.313157:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04277 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760700.313241:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493635968:12345-192.168.203.60@tcp:4:dd.0 Request processed in 78846us (79076us total) trans 4294972450 rc 0/0 00000100:00100000:2.0:1789760700.313251:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5161 00000100:00100000:2.0:1789760700.313262:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710502640194 time 710575415070 state 0x2 ref 0 00000100:00100000:1.0:1789760700.321077:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760700.321210:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493636096 00000100:00100000:2.0:1789760700.321222:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493636096 00000100:00100000:2.0:1789760700.321281:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710502640194 time 710575415070 state 0x0 ref 1 00000100:00100000:2.0:1789760700.321299:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710575675486 time 710575415070 state 0x1 ref 1 00000100:00100000:2.0:1789760700.321331:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710575675486 time 710662365804 state 0x0 ref 1 00000100:00100000:2.0:1789760700.321344:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493636096:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760700.321399:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21020672, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa807 00010000:00010000:2.0:1789760700.321408:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0427e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.321424:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760700.321430:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760700.321436:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0427e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.321451:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0427e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760700.420908:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972451 is committed 00002000:00010000:1.0:1789760700.421067:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21020672 -> 21024768 00010000:00010000:1.0:1789760700.421082:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0427e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.421096:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0427e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.421112:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0427e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.421120:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760700.421125:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0427e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.421135:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0427e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.421156:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0427e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.421165:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0427e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760700.421261:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493636096:12345-192.168.203.60@tcp:4:dd.0 Request processed in 99915us (100188us total) trans 4294972451 rc 0/0 00000100:00100000:1.0:1789760700.424588:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5162 00000100:00100000:1.0:1789760700.424602:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710575675486 time 710662365804 state 0x2 ref 0 00000100:00100000:1.0:1789760700.427258:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760700.427361:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493636224 00000100:00100000:2.0:1789760700.427372:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493636224 00000100:00100000:2.0:1789760700.427499:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710575675486 time 710662365804 state 0x0 ref 1 00000100:00100000:2.0:1789760700.427516:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710662626220 time 710662365804 state 0x1 ref 1 00000100:00100000:2.0:1789760700.427551:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710662626220 time 710768586874 state 0x0 ref 1 00000100:00100000:2.0:1789760700.427564:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493636224:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760700.427880:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21024768, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa807 00010000:00010000:2.0:1789760700.428076:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04285 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.428091:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760700.428096:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760700.428102:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04285 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.428117:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04285 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760700.614496:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21024768 -> 21028864 00010000:00010000:1.0:1789760700.614515:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04285 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.614534:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04285 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.614543:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04285 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.614551:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760700.614556:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04285 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.614564:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04285 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.614585:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04285 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.614592:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04285 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760700.614720:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493636224:12345-192.168.203.60@tcp:4:dd.0 Request processed in 187154us (187465us total) trans 4294972452 rc 0/0 00000100:00100000:1.0:1789760700.614731:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5163 00000100:00100000:1.0:1789760700.614741:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710662626220 time 710768586874 state 0x2 ref 0 00000001:00080000:2.0:1789760700.614946:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972452 is committed 00000100:00100000:3.0:1789760700.619898:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760700.619978:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493636352 00000100:00100000:1.0:1789760700.619989:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493636352 00000100:00100000:1.0:1789760700.620044:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710662626220 time 710768586874 state 0x0 ref 1 00000100:00100000:1.0:1789760700.620060:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710768847290 time 710768586874 state 0x1 ref 1 00000100:00100000:1.0:1789760700.620094:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710768847290 time 710961130036 state 0x0 ref 1 00000100:00100000:1.0:1789760700.620107:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493636352:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760700.620153:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21028864, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0xa807 00010000:00010000:1.0:1789760700.620162:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0428c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.620176:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760700.620180:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760700.620186:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0428c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.620200:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0428c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760700.651737:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972453 is committed 00002000:00010000:1.0:1789760700.652010:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21028864 -> 21032960 00002000:00010000:1.0:1789760700.652015:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760699 -> 1789760700 00002000:00010000:1.0:1789760700.652018:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760699 -> 1789760700 00010000:00010000:1.0:1789760700.652031:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0428c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.652045:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0428c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.652054:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0428c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.652062:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760700.652067:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0428c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.652075:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0428c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.652097:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0428c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.652106:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0428c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760700.652200:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493636352:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32091us (32308us total) trans 4294972453 rc 0/0 00000100:00100000:1.0:1789760700.652211:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5164 00000100:00100000:1.0:1789760700.652220:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710768847290 time 710961130036 state 0x2 ref 0 00000100:00100000:2.0:1789760700.659694:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760700.659810:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493636480 00000100:00100000:0.0:1789760700.659820:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493636480 00000100:00100000:0.0:1789760700.659890:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710768847290 time 710961130036 state 0x0 ref 1 00000100:00100000:0.0:1789760700.659906:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710961390452 time 710961130036 state 0x1 ref 1 00000100:00100000:0.0:1789760700.659937:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710961390452 time 711000972793 state 0x0 ref 1 00000100:00100000:0.0:1789760700.659951:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493636480:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760700.660180:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21032960, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa807 00010000:00010000:0.0:1789760700.660189:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04293 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.660205:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760700.660210:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760700.660216:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04293 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.660237:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04293 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760700.809670:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972454 is committed 00002000:00010000:0.0:1789760700.809742:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21032960 -> 21037056 00010000:00010000:0.0:1789760700.809759:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04293 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.809772:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04293 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.809793:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04293 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.809801:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760700.809806:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04293 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.809815:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04293 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.809844:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04293 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.809852:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04293 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760700.809934:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493636480:12345-192.168.203.60@tcp:4:dd.0 Request processed in 149981us (150248us total) trans 4294972454 rc 0/0 00000100:00100000:0.0:1789760700.809944:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5165 00000100:00100000:0.0:1789760700.809954:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710961390452 time 711000972793 state 0x2 ref 0 00000100:00100000:0.0:1789760700.813907:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760700.814250:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493636608 00000100:00100000:1.0:1789760700.814261:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493636608 00000100:00100000:1.0:1789760700.814315:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 710961390452 time 711000972793 state 0x0 ref 1 00000100:00100000:1.0:1789760700.814330:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711001233209 time 711000972793 state 0x1 ref 1 00000100:00100000:1.0:1789760700.814365:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711001233209 time 711155399596 state 0x0 ref 1 00000100:00100000:1.0:1789760700.814378:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493636608:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760700.814440:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21037056, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa807 00010000:00010000:1.0:1789760700.814449:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0429a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.814463:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760700.814468:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760700.814473:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0429a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760700.814487:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0429a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.953549:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760700.953804:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972455 is committed 00002000:00010000:0.0:1789760700.954023:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21037056 -> 21041152 00010000:00010000:0.0:1789760700.954038:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0429a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.954051:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0429a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.954062:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0429a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.954069:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760700.954074:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0429a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.954083:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0429a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.954100:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0429a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.954108:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0429a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760700.954207:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493636608:12345-192.168.203.60@tcp:4:dd.0 Request processed in 139827us (140305us total) trans 4294972455 rc 0/0 00000100:00100000:0.0:1789760700.954218:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5166 00000100:00100000:0.0:1789760700.954228:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711001233209 time 711155399596 state 0x2 ref 0 00000100:00100000:0.0:1789760700.964471:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760700.964573:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493636736 00000100:00100000:2.0:1789760700.964583:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493636736 00000100:00100000:2.0:1789760700.964638:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711001233209 time 711155399596 state 0x0 ref 1 00000100:00100000:2.0:1789760700.964653:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711155660012 time 711155399596 state 0x1 ref 1 00000100:00100000:2.0:1789760700.964682:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711155660012 time 711305717367 state 0x0 ref 1 00000100:00100000:2.0:1789760700.964693:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493636736:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760700.964739:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21041152, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa807 00010000:00010000:2.0:1789760700.964748:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e042a1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.964763:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760700.964768:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760700.964774:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e042a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760700.964800:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e042a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760701.113228:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972456 is committed 00002000:00010000:3.0:1789760701.113395:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21041152 -> 21045248 00010000:00010000:3.0:1789760701.113410:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e042a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.113429:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e042a1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.113438:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e042a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.113449:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760701.113453:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e042a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.113464:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e042a1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.113483:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e042a1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.113491:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e042a1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760701.113587:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493636736:12345-192.168.203.60@tcp:4:dd.0 Request processed in 148891us (149123us total) trans 4294972456 rc 0/0 00000100:00100000:3.0:1789760701.113598:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5167 00000100:00100000:3.0:1789760701.113608:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711155660012 time 711305717367 state 0x2 ref 0 00000100:00100000:1.0:1789760701.121749:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760701.121844:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493636864 00000100:00100000:3.0:1789760701.121855:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493636864 00000100:00100000:3.0:1789760701.121934:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711155660012 time 711305717367 state 0x0 ref 1 00000100:00100000:3.0:1789760701.121950:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711305977783 time 711305717367 state 0x1 ref 1 00000100:00100000:3.0:1789760701.122012:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711305977783 time 711463047185 state 0x0 ref 1 00000100:00100000:3.0:1789760701.122026:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493636864:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760701.122078:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21045248, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa807 00010000:00010000:3.0:1789760701.122087:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e042a8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.122103:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760701.122108:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760701.122115:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e042a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.122127:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e042a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760701.122891:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760701.189463:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972457 is committed 00002000:00010000:0.0:1789760701.189523:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21045248 -> 21049344 00010000:00010000:0.0:1789760701.189537:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e042a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.189549:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e042a8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.189558:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e042a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.190032:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760701.190037:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e042a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.190054:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e042a8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.190074:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e042a8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.190083:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e042a8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.192035:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493636864:12345-192.168.203.60@tcp:4:dd.0 Request processed in 70000us (70286us total) trans 4294972457 rc 0/0 00000100:00100000:1.0:1789760701.192050:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5168 00000100:00100000:1.0:1789760701.192060:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711305977783 time 711463047185 state 0x2 ref 0 00000100:00100000:0.0:1789760701.203768:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760701.203923:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493636992 00000100:00100000:2.0:1789760701.203932:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493636992 00000100:00100000:2.0:1789760701.204023:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711305977783 time 711463047185 state 0x0 ref 1 00000100:00100000:2.0:1789760701.204040:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711463307601 time 711463047185 state 0x1 ref 1 00000100:00100000:2.0:1789760701.204071:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711463307601 time 711545105902 state 0x0 ref 1 00000100:00100000:2.0:1789760701.204084:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493636992:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760701.204134:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21049344, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa807 00010000:00010000:2.0:1789760701.204144:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e042af lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.204161:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760701.204165:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760701.204171:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e042af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.204184:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e042af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760701.239697:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972458 is committed 00002000:00010000:2.0:1789760701.240251:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21049344 -> 21053440 00010000:00010000:2.0:1789760701.240354:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e042af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.240368:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e042af lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.240378:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e042af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.240386:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760701.240391:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e042af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.240401:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e042af lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.240421:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e042af lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.240428:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e042af lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760701.240523:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493636992:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36437us (36761us total) trans 4294972458 rc 0/0 00000100:00100000:2.0:1789760701.240532:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5169 00000100:00100000:2.0:1789760701.240542:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711463307601 time 711545105902 state 0x2 ref 0 00000100:00100000:2.0:1789760701.245016:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760701.245091:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493637120 00000100:00100000:0.0:1789760701.245101:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493637120 00000100:00100000:0.0:1789760701.245151:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711463307601 time 711545105902 state 0x0 ref 1 00000100:00100000:0.0:1789760701.245167:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711545366318 time 711545105902 state 0x1 ref 1 00000100:00100000:0.0:1789760701.245197:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711545366318 time 711586232846 state 0x0 ref 1 00000100:00100000:0.0:1789760701.245208:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493637120:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760701.245256:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21053440, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa807 00010000:00010000:0.0:1789760701.245273:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042b6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.245290:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760701.245296:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760701.245303:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.245317:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760701.301202:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972459 is committed 00002000:00010000:0.0:1789760701.301354:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21053440 -> 21057536 00010000:00010000:0.0:1789760701.301369:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.301384:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042b6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.301393:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.301401:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760701.301406:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.301415:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042b6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.301433:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042b6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.301441:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042b6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760701.301539:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493637120:12345-192.168.203.60@tcp:4:dd.0 Request processed in 56327us (56654us total) trans 4294972459 rc 0/0 00000100:00100000:0.0:1789760701.301549:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5170 00000100:00100000:0.0:1789760701.301559:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711545366318 time 711586232846 state 0x2 ref 0 00000100:00100000:1.0:1789760701.305919:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760701.306172:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493637248 00000100:00100000:3.0:1789760701.306183:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493637248 00000100:00100000:3.0:1789760701.306240:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711545366318 time 711586232846 state 0x0 ref 1 00000100:00100000:3.0:1789760701.306269:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711586493262 time 711586232846 state 0x1 ref 1 00000100:00100000:3.0:1789760701.306305:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711586493262 time 711647340009 state 0x0 ref 1 00000100:00100000:3.0:1789760701.306318:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493637248:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760701.306369:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21057536, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa807 00010000:00010000:3.0:1789760701.306379:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e042bd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.306394:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760701.306399:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760701.306406:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e042bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.306422:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e042bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760701.439080:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972460 is committed 00002000:00010000:3.0:1789760701.439250:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21057536 -> 21061632 00010000:00010000:3.0:1789760701.439276:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e042bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.439290:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e042bd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.439299:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e042bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.439307:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760701.439312:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e042bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.439322:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e042bd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.439341:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e042bd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.439349:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e042bd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760701.439443:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493637248:12345-192.168.203.60@tcp:4:dd.0 Request processed in 133122us (133528us total) trans 4294972460 rc 0/0 00000100:00100000:3.0:1789760701.439454:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5171 00000100:00100000:3.0:1789760701.439464:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711586493262 time 711647340009 state 0x2 ref 0 00000100:00100000:0.0:1789760701.447308:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760701.447452:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493637376 00000100:00100000:1.0:1789760701.447463:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493637376 00000100:00100000:1.0:1789760701.447518:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711586493262 time 711647340009 state 0x0 ref 1 00000100:00100000:1.0:1789760701.447535:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711647600425 time 711647340009 state 0x1 ref 1 00000100:00100000:1.0:1789760701.447565:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711647600425 time 711788599771 state 0x0 ref 1 00000100:00100000:1.0:1789760701.447587:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493637376:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760701.447636:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21061632, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa807 00010000:00010000:1.0:1789760701.447645:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e042c4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.447662:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760701.447667:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760701.447674:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e042c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.447688:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e042c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760701.501650:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972461 is committed 00002000:00010000:1.0:1789760701.501740:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21061632 -> 21065728 00010000:00010000:1.0:1789760701.501755:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e042c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.501769:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e042c4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.501788:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e042c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.501797:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760701.501802:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e042c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.501811:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e042c4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.501958:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e042c4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.501990:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e042c4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.502130:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493637376:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54541us (54826us total) trans 4294972461 rc 0/0 00000100:00100000:1.0:1789760701.502141:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5172 00000100:00100000:1.0:1789760701.502151:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711647600425 time 711788599771 state 0x2 ref 0 00000100:00100000:2.0:1789760701.506700:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760701.507478:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493637504 00000100:00100000:0.0:1789760701.507490:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493637504 00000100:00100000:0.0:1789760701.507547:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711647600425 time 711788599771 state 0x0 ref 1 00000100:00100000:0.0:1789760701.507563:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711788860187 time 711788599771 state 0x1 ref 1 00000100:00100000:0.0:1789760701.507633:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711788860187 time 711848668145 state 0x0 ref 1 00000100:00100000:0.0:1789760701.507647:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493637504:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760701.507700:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21065728, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa807 00010000:00010000:0.0:1789760701.507710:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042cb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.507727:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760701.507733:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760701.507739:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.507756:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760701.546175:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760701.546226:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760701.546252:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760701.546349:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493637760 00000100:00100000:3.0:1789760701.546359:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493637760 00000100:00100000:3.0:1789760701.546418:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493637888 00000100:00100000:3.0:1789760701.546421:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493637888 00000100:00100000:3.0:1789760701.546447:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 442 00000100:00100000:3.0:1789760701.546456:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2369:x1876699493637760:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:3.0:1789760701.546523:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2369:x1876699493637760:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 68us (358us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760701.546531:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 442 00000100:00100000:3.0:1789760701.546547:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 443 00000100:00100000:3.0:1789760701.546554:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2369:x1876699493637888:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:3.0:1789760701.546577:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2369:x1876699493637888:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 24us (325us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760701.546583:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 443 00000100:00080000:0.0:1789760701.548433:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760701.554008:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493637632 00000100:00100000:1.0:1789760701.554027:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493637632 00000100:00100000:1.0:1789760701.554088:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 383 00000100:00100000:1.0:1789760701.554098:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7bfe450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2369:x1876699493637632:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:1.0:1789760701.554198:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7bfe450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2369:x1876699493637632:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 100us (7970us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760701.554206:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 383 00000001:00080000:2.0:1789760701.571797:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972462 is committed 00002000:00010000:1.0:1789760701.572033:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21065728 -> 21069824 00010000:00010000:1.0:1789760701.572052:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.572065:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042cb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.572074:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.572081:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760701.572087:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.572096:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042cb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.572117:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042cb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.572138:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042cb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.572247:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493637504:12345-192.168.203.60@tcp:4:dd.0 Request processed in 64595us (65550us total) trans 4294972462 rc 0/0 00000100:00100000:1.0:1789760701.572270:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5173 00000100:00100000:1.0:1789760701.572281:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711788860187 time 711848668145 state 0x2 ref 0 00000100:00100000:3.0:1789760701.579345:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760701.579576:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493638016 00000100:00100000:0.0:1789760701.579589:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493638016 00000100:00100000:0.0:1789760701.579646:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711788860187 time 711848668145 state 0x0 ref 1 00000100:00100000:0.0:1789760701.579662:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711848928561 time 711848668145 state 0x1 ref 1 00000100:00100000:0.0:1789760701.579694:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711848928561 time 711920729953 state 0x0 ref 1 00000100:00100000:0.0:1789760701.579706:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493638016:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760701.579764:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21069824, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0xa807 00010000:00010000:0.0:1789760701.579773:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e042d2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.579799:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760701.579804:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760701.579810:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e042d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.579824:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e042d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760701.655486:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21069824 -> 21073920 00002000:00010000:0.0:1789760701.655496:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760700 -> 1789760701 00002000:00010000:0.0:1789760701.655499:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760700 -> 1789760701 00010000:00010000:0.0:1789760701.655512:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e042d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.655526:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e042d2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.655536:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e042d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.655543:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760701.655548:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e042d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.655558:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e042d2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.655577:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e042d2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.655585:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e042d2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760701.655696:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493638016:12345-192.168.203.60@tcp:4:dd.0 Request processed in 75986us (76356us total) trans 4294972463 rc 0/0 00000100:00100000:0.0:1789760701.655707:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5174 00000100:00100000:0.0:1789760701.655725:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711848928561 time 711920729953 state 0x2 ref 0 00000001:00080000:3.0:1789760701.655936:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972463 is committed 00000100:00100000:0.0:1789760701.661388:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760701.663942:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493638144 00000100:00100000:1.0:1789760701.663957:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493638144 00000100:00100000:1.0:1789760701.664061:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711848928561 time 711920729953 state 0x0 ref 1 00000100:00100000:1.0:1789760701.664077:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711920990369 time 711920729953 state 0x1 ref 1 00000100:00100000:1.0:1789760701.664106:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711920990369 time 712005141344 state 0x0 ref 1 00000100:00100000:1.0:1789760701.664120:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493638144:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760701.664170:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21073920, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:1.0:1789760701.664179:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e042d9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.664199:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760701.664204:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760701.664210:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e042d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.664226:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e042d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760701.718075:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972464 is committed 00002000:00010000:1.0:1789760701.720355:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21073920 -> 21078016 00010000:00010000:1.0:1789760701.720377:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e042d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.720390:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e042d9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.720400:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e042d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.720408:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760701.720676:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e042d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.720687:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e042d9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.720708:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e042d9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.720716:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e042d9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.720875:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493638144:12345-192.168.203.60@tcp:4:dd.0 Request processed in 56753us (59494us total) trans 4294972464 rc 0/0 00000100:00100000:1.0:1789760701.720886:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5175 00000100:00100000:1.0:1789760701.720895:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711920990369 time 712005141344 state 0x2 ref 0 00000100:00100000:2.0:1789760701.725445:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760701.725511:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493638272 00000100:00100000:3.0:1789760701.725521:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493638272 00000100:00100000:3.0:1789760701.725572:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 711920990369 time 712005141344 state 0x0 ref 1 00000100:00100000:3.0:1789760701.725588:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712005401760 time 712005141344 state 0x1 ref 1 00000100:00100000:3.0:1789760701.725617:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712005401760 time 712066652276 state 0x0 ref 1 00000100:00100000:3.0:1789760701.725630:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493638272:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760701.725678:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21078016, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:3.0:1789760701.725687:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e042e0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.725703:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760701.725708:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760701.725714:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e042e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760701.725729:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e042e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760701.793148:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972465 is committed 00002000:00010000:1.0:1789760701.796026:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21078016 -> 21082112 00010000:00010000:1.0:1789760701.796047:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e042e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.796067:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e042e0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.796076:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e042e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.796084:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760701.796089:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e042e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.796099:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e042e0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.796119:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e042e0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.796126:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e042e0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.796220:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493638272:12345-192.168.203.60@tcp:4:dd.0 Request processed in 70588us (70779us total) trans 4294972465 rc 0/0 00000100:00100000:1.0:1789760701.796231:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5176 00000100:00100000:1.0:1789760701.796242:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712005401760 time 712066652276 state 0x2 ref 0 00000100:00100000:1.0:1789760701.802228:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760701.802345:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493638400 00000100:00100000:0.0:1789760701.802356:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493638400 00000100:00100000:0.0:1789760701.802407:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712005401760 time 712066652276 state 0x0 ref 1 00000100:00100000:0.0:1789760701.802423:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712066912692 time 712066652276 state 0x1 ref 1 00000100:00100000:0.0:1789760701.802454:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712066912692 time 712143489175 state 0x0 ref 1 00000100:00100000:0.0:1789760701.802466:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493638400:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760701.802517:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21082112, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:0.0:1789760701.802526:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042e7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.802542:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760701.802547:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760701.802553:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.802567:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760701.890737:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21082112 -> 21086208 00010000:00010000:0.0:1789760701.890756:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.890770:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042e7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.890790:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.890799:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760701.890803:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.890813:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042e7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.890838:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042e7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.890846:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e042e7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760701.890938:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493638400:12345-192.168.203.60@tcp:4:dd.0 Request processed in 88469us (88718us total) trans 4294972466 rc 0/0 00000100:00100000:0.0:1789760701.890952:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5177 00000100:00100000:0.0:1789760701.890997:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712066912692 time 712143489175 state 0x2 ref 0 00000001:00080000:0.0:1789760701.892049:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972466 is committed 00000100:00100000:0.0:1789760701.895399:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760701.895944:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493638528 00000100:00100000:2.0:1789760701.895955:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493638528 00000100:00100000:2.0:1789760701.896040:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712066912692 time 712143489175 state 0x0 ref 1 00000100:00100000:2.0:1789760701.896057:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712143749591 time 712143489175 state 0x1 ref 1 00000100:00100000:2.0:1789760701.896701:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712143749591 time 712237732981 state 0x0 ref 1 00000100:00100000:2.0:1789760701.896718:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493638528:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760701.896767:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21086208, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:2.0:1789760701.896787:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e042ee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.896804:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760701.896809:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760701.896815:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e042ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.896839:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e042ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760701.955916:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972467 is committed 00002000:00010000:1.0:1789760701.956038:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21086208 -> 21090304 00010000:00010000:1.0:1789760701.956058:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e042ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.956074:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e042ee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.956085:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e042ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.956093:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760701.956098:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e042ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.956111:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e042ee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.956133:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e042ee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760701.956140:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e042ee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760701.956238:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493638528:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59514us (60838us total) trans 4294972467 rc 0/0 00000100:00100000:1.0:1789760701.956249:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5178 00000100:00100000:1.0:1789760701.956258:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712143749591 time 712237732981 state 0x2 ref 0 00000100:00100000:1.0:1789760701.959843:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760701.959938:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493638656 00000100:00100000:2.0:1789760701.959948:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493638656 00000100:00100000:2.0:1789760701.960040:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712143749591 time 712237732981 state 0x0 ref 1 00000100:00100000:2.0:1789760701.960057:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712237993397 time 712237732981 state 0x1 ref 1 00000100:00100000:2.0:1789760701.960077:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712237993397 time 712301113151 state 0x0 ref 1 00000100:00100000:2.0:1789760701.960089:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493638656:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760701.960138:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21090304, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:2.0:1789760701.960148:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e042f5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.960163:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760701.960168:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760701.960174:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e042f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760701.960187:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e042f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.015155:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760702.024301:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21090304 -> 21094400 00010000:00010000:2.0:1789760702.024321:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e042f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.024334:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e042f5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.024343:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e042f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.024351:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760702.024355:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e042f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.024363:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e042f5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.024383:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e042f5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.024392:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e042f5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760702.024489:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493638656:12345-192.168.203.60@tcp:4:dd.0 Request processed in 64397us (64651us total) trans 4294972468 rc 0/0 00000100:00100000:2.0:1789760702.024500:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5179 00000100:00100000:2.0:1789760702.024509:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712237993397 time 712301113151 state 0x2 ref 0 00000001:00080000:0.0:1789760702.025106:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972468 is committed 00000100:00100000:0.0:1789760702.029429:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760702.029514:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493638784 00000100:00100000:3.0:1789760702.029523:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493638784 00000100:00100000:3.0:1789760702.029579:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712237993397 time 712301113151 state 0x0 ref 1 00000100:00100000:3.0:1789760702.029596:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712301373567 time 712301113151 state 0x1 ref 1 00000100:00100000:3.0:1789760702.029628:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712301373567 time 712370663320 state 0x0 ref 1 00000100:00100000:3.0:1789760702.029641:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493638784:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760702.029689:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21094400, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:3.0:1789760702.029698:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e042fc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.029714:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760702.029719:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760702.029725:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e042fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.029739:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e042fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760702.062376:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972469 is committed 00002000:00010000:2.0:1789760702.062534:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21094400 -> 21098496 00010000:00010000:2.0:1789760702.062548:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e042fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.062561:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e042fc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.062570:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e042fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.062576:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760702.062581:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e042fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.062596:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e042fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.062617:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e042fc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.062625:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e042fc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760702.062748:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493638784:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33104us (33322us total) trans 4294972469 rc 0/0 00000100:00100000:2.0:1789760702.062758:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5180 00000100:00100000:2.0:1789760702.062767:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712301373567 time 712370663320 state 0x2 ref 0 00000100:00100000:3.0:1789760702.065609:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760702.065730:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493638912 00000100:00100000:0.0:1789760702.065740:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493638912 00000100:00100000:0.0:1789760702.065867:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712301373567 time 712370663320 state 0x0 ref 1 00000100:00100000:0.0:1789760702.065882:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712370923736 time 712370663320 state 0x1 ref 1 00000100:00100000:0.0:1789760702.065916:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712370923736 time 712406951425 state 0x0 ref 1 00000100:00100000:0.0:1789760702.065929:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493638912:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760702.066015:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21098496, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:0.0:1789760702.066024:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04303 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.066039:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760702.066043:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760702.066050:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04303 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.066068:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04303 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760702.103445:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972470 is committed 00002000:00010000:2.0:1789760702.103721:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21098496 -> 21102592 00010000:00010000:2.0:1789760702.103737:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04303 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.103749:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04303 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.103757:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04303 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.103764:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760702.103769:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04303 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.103789:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04303 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.103807:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04303 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.103815:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04303 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760702.103909:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493638912:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37979us (38307us total) trans 4294972470 rc 0/0 00000100:00100000:2.0:1789760702.103919:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5181 00000100:00100000:2.0:1789760702.103929:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712370923736 time 712406951425 state 0x2 ref 0 00000100:00100000:1.0:1789760702.113820:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760702.114001:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493639040 00000100:00100000:2.0:1789760702.114012:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493639040 00000100:00100000:2.0:1789760702.114068:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712370923736 time 712406951425 state 0x0 ref 1 00000100:00100000:2.0:1789760702.114083:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712407211841 time 712406951425 state 0x1 ref 1 00000100:00100000:2.0:1789760702.114113:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712407211841 time 712455149019 state 0x0 ref 1 00000100:00100000:2.0:1789760702.114125:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493639040:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760702.114173:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21102592, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:2.0:1789760702.114183:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0430a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.114199:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760702.114204:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760702.114210:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0430a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.114224:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0430a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760702.169649:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21102592 -> 21106688 00010000:00010000:2.0:1789760702.169669:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0430a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.169683:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0430a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.169693:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0430a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.169701:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760702.169707:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0430a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.169717:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0430a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.169743:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0430a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.169752:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0430a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760702.169869:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493639040:12345-192.168.203.60@tcp:4:dd.0 Request processed in 55739us (56053us total) trans 4294972471 rc 0/0 00000100:00100000:2.0:1789760702.169879:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5182 00000100:00100000:2.0:1789760702.169889:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712407211841 time 712455149019 state 0x2 ref 0 00000001:00080000:3.0:1789760702.173358:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972471 is committed 00000100:00100000:0.0:1789760702.173511:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760702.173584:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493639168 00000100:00100000:2.0:1789760702.173593:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493639168 00000100:00100000:2.0:1789760702.173647:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712407211841 time 712455149019 state 0x0 ref 1 00000100:00100000:2.0:1789760702.173661:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712455409435 time 712455149019 state 0x1 ref 1 00000100:00100000:2.0:1789760702.173695:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712455409435 time 712514730275 state 0x0 ref 1 00000100:00100000:2.0:1789760702.173709:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493639168:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760702.173755:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21106688, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:2.0:1789760702.173765:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04311 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.173799:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760702.173805:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760702.173811:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04311 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.173833:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04311 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760702.205431:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21106688 -> 21110784 00010000:00010000:2.0:1789760702.205450:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04311 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.205463:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04311 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.205471:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04311 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.205478:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760702.205483:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04311 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.205492:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04311 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.205511:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04311 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.205524:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04311 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760702.205621:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493639168:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31911us (32111us total) trans 4294972472 rc 0/0 00000100:00100000:2.0:1789760702.205632:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5183 00000100:00100000:2.0:1789760702.205644:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712455409435 time 712514730275 state 0x2 ref 0 00000001:00080000:0.0:1789760702.205721:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972472 is committed 00000100:00100000:0.0:1789760702.208925:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760702.209296:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493639296 00000100:00100000:3.0:1789760702.209306:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493639296 00000100:00100000:3.0:1789760702.209363:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712455409435 time 712514730275 state 0x0 ref 1 00000100:00100000:3.0:1789760702.209380:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712514990691 time 712514730275 state 0x1 ref 1 00000100:00100000:3.0:1789760702.209442:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712514990691 time 712550477792 state 0x0 ref 1 00000100:00100000:3.0:1789760702.209455:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493639296:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760702.209503:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21110784, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:3.0:1789760702.209512:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04318 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.209527:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760702.209531:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760702.209538:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04318 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.209552:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04318 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760702.241244:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972473 is committed 00002000:00010000:2.0:1789760702.241384:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21110784 -> 21114880 00010000:00010000:2.0:1789760702.241403:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04318 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.241415:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04318 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.241424:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04318 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.241431:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760702.241435:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04318 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.241444:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04318 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.241468:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04318 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.241476:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04318 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760702.241604:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493639296:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32146us (32683us total) trans 4294972473 rc 0/0 00000100:00100000:2.0:1789760702.241615:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5184 00000100:00100000:2.0:1789760702.241625:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712514990691 time 712550477792 state 0x2 ref 0 00000100:00100000:1.0:1789760702.245200:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760702.245735:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493639424 00000100:00100000:2.0:1789760702.245745:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493639424 00000100:00100000:2.0:1789760702.245811:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712514990691 time 712550477792 state 0x0 ref 1 00000100:00100000:2.0:1789760702.245887:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712550738208 time 712550477792 state 0x1 ref 1 00000100:00100000:2.0:1789760702.245995:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712550738208 time 712587029408 state 0x0 ref 1 00000100:00100000:2.0:1789760702.246009:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493639424:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760702.246061:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21114880, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:2.0:1789760702.246069:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0431f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.246085:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760702.246089:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760702.246095:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0431f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.246108:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0431f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760702.283511:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972474 is committed 00002000:00010000:1.0:1789760702.283620:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21114880 -> 21118976 00010000:00010000:1.0:1789760702.283636:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0431f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.283648:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0431f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.283657:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0431f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.283664:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760702.283668:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0431f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.283676:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0431f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.283695:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0431f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.283703:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0431f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.283811:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493639424:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37800us (38615us total) trans 4294972474 rc 0/0 00000100:00100000:1.0:1789760702.283821:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5185 00000100:00100000:1.0:1789760702.284299:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712550738208 time 712587029408 state 0x2 ref 0 00000100:00100000:0.0:1789760702.289292:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760702.289534:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493639552 00000100:00100000:2.0:1789760702.289545:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493639552 00000100:00100000:2.0:1789760702.289601:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712550738208 time 712587029408 state 0x0 ref 1 00000100:00100000:2.0:1789760702.289617:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712587289824 time 712587029408 state 0x1 ref 1 00000100:00100000:2.0:1789760702.289648:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712587289824 time 712630683579 state 0x0 ref 1 00000100:00100000:2.0:1789760702.289660:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493639552:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760702.289708:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21118976, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:2.0:1789760702.289723:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04326 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.289743:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760702.289751:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760702.289757:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04326 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.289780:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04326 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.330109:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc5837100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3303:1876699519245184:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760702.330178:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760702.330242:0:6691:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519245184 00000100:00100000:2.0:1789760702.330375:0:6691:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519245184 00000100:00100000:2.0:1789760702.330737:0:6691:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 184 00000100:00100000:2.0:1789760702.330748:0:6691:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876699519245184:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760702.330782:0:6691:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527630336 pending 1703936 free 3836739584 avail 3812622336 00000100:00100000:2.0:1789760702.331062:0:6691:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876699519245184:12345-0@lo:13:osp-pre-0-0.0 Request processed in 315us (883us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760702.331070:0:6691:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 184 00000100:00100000:1.0:1789760702.331721:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc5837100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3303:1876699519245184:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1789760702.332455:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789760702.336722:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21118976 -> 21123072 00010000:00010000:1.0:1789760702.336743:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04326 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.336755:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04326 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.336763:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04326 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.336783:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760702.336789:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04326 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.336797:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04326 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.336821:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04326 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.336829:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04326 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.336921:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493639552:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47257us (47634us total) trans 4294972475 rc 0/0 00000100:00100000:1.0:1789760702.336932:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5186 00000100:00100000:1.0:1789760702.336942:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712587289824 time 712630683579 state 0x2 ref 0 00000001:00080000:0.0:1789760702.338068:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972475 is committed 00000100:00100000:2.0:1789760702.345041:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760702.345128:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493639680 00000100:00100000:0.0:1789760702.345137:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493639680 00000100:00100000:0.0:1789760702.345189:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712587289824 time 712630683579 state 0x0 ref 1 00000100:00100000:0.0:1789760702.345205:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712630943995 time 712630683579 state 0x1 ref 1 00000100:00100000:0.0:1789760702.345234:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712630943995 time 712686269625 state 0x0 ref 1 00000100:00100000:0.0:1789760702.345247:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493639680:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760702.345297:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21123072, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:0.0:1789760702.346878:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0432d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.346896:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760702.346901:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760702.346907:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0432d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.346928:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0432d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760702.418425:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972476 is committed 00002000:00010000:1.0:1789760702.419194:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21123072 -> 21127168 00010000:00010000:1.0:1789760702.419209:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0432d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.419222:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0432d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.419232:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0432d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.419239:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760702.419243:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0432d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.419257:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0432d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.419275:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0432d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.419283:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0432d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.419383:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493639680:12345-192.168.203.60@tcp:4:dd.0 Request processed in 74132us (74346us total) trans 4294972476 rc 0/0 00000100:00100000:1.0:1789760702.419393:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5187 00000100:00100000:1.0:1789760702.419404:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712630943995 time 712686269625 state 0x2 ref 0 00000100:00100000:1.0:1789760702.423069:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760702.423138:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493639808 00000100:00100000:3.0:1789760702.423148:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493639808 00000100:00100000:3.0:1789760702.423214:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712630943995 time 712686269625 state 0x0 ref 1 00000100:00100000:3.0:1789760702.423229:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712686530041 time 712686269625 state 0x1 ref 1 00000100:00100000:3.0:1789760702.423259:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712686530041 time 712764294367 state 0x0 ref 1 00000100:00100000:3.0:1789760702.423272:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493639808:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760702.423332:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21127168, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:3.0:1789760702.423342:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04334 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.423357:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760702.423362:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760702.423368:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04334 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.423381:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04334 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760702.478636:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972477 is committed 00002000:00010000:3.0:1789760702.478759:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21127168 -> 21131264 00010000:00010000:3.0:1789760702.478783:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04334 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.478796:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04334 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.478804:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04334 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.478816:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760702.478820:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04334 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.479031:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04334 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.479055:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04334 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.479064:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04334 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760702.479190:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493639808:12345-192.168.203.60@tcp:4:dd.0 Request processed in 55916us (56125us total) trans 4294972477 rc 0/0 00000100:00100000:3.0:1789760702.479201:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5188 00000100:00100000:3.0:1789760702.479211:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712686530041 time 712764294367 state 0x2 ref 0 00000100:00100000:3.0:1789760702.487798:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760702.487911:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493639936 00000100:00100000:1.0:1789760702.487921:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493639936 00000100:00100000:1.0:1789760702.488011:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712686530041 time 712764294367 state 0x0 ref 1 00000100:00100000:1.0:1789760702.488025:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712764554783 time 712764294367 state 0x1 ref 1 00000100:00100000:1.0:1789760702.488058:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712764554783 time 712829092982 state 0x0 ref 1 00000100:00100000:1.0:1789760702.488071:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493639936:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760702.488125:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21131264, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:1.0:1789760702.488135:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0433b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.488151:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760702.488155:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.488161:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0433b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.488175:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0433b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760702.542607:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972478 is committed 00002000:00010000:1.0:1789760702.542701:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21131264 -> 21135360 00010000:00010000:1.0:1789760702.542717:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0433b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.542730:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0433b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.542739:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0433b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.542747:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760702.542752:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0433b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.542763:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0433b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.542793:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0433b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.542800:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0433b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760702.542900:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493639936:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54827us (55109us total) trans 4294972478 rc 0/0 00000100:00100000:1.0:1789760702.542910:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5189 00000100:00100000:1.0:1789760702.542920:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712764554783 time 712829092982 state 0x2 ref 0 00000100:00100000:2.0:1789760702.549841:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760702.550023:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493640064 00000100:00100000:1.0:1789760702.550034:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493640064 00000100:00100000:1.0:1789760702.550085:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712764554783 time 712829092982 state 0x0 ref 1 00000100:00100000:1.0:1789760702.550098:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712829353398 time 712829092982 state 0x1 ref 1 00000100:00100000:1.0:1789760702.550125:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712829353398 time 712891161602 state 0x0 ref 1 00000100:00100000:1.0:1789760702.550136:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493640064:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760702.550180:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21135360, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:1.0:1789760702.550188:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04342 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.550202:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760702.550207:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.550213:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04342 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.550225:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04342 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760702.599602:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972479 is committed 00002000:00010000:2.0:1789760702.599711:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21135360 -> 21139456 00010000:00010000:2.0:1789760702.599728:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04342 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.599741:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04342 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.599750:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04342 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.599758:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760702.599763:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04342 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.599790:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04342 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.599810:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04342 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.599818:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04342 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760702.599926:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493640064:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49786us (50099us total) trans 4294972479 rc 0/0 00000100:00100000:2.0:1789760702.599936:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5190 00000100:00100000:2.0:1789760702.599947:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712829353398 time 712891161602 state 0x2 ref 0 00000100:00100000:2.0:1789760702.604496:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760702.604595:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493640192 00000100:00100000:1.0:1789760702.604605:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493640192 00000100:00100000:1.0:1789760702.604660:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712829353398 time 712891161602 state 0x0 ref 1 00000100:00100000:1.0:1789760702.604676:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712891422018 time 712891161602 state 0x1 ref 1 00000100:00100000:1.0:1789760702.604735:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712891422018 time 712945769785 state 0x0 ref 1 00000100:00100000:1.0:1789760702.604749:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493640192:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760702.604809:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21139456, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0xa807 00010000:00010000:1.0:1789760702.604820:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04349 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.604846:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760702.604850:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.604855:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04349 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.604870:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04349 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760702.649028:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972480 is committed 00002000:00010000:2.0:1789760702.649500:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21139456 -> 21143552 00002000:00010000:2.0:1789760702.649505:0:13038:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760701 -> 1789760702 00002000:00010000:2.0:1789760702.649508:0:13038:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760701 -> 1789760702 00010000:00010000:2.0:1789760702.649522:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04349 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.649535:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04349 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.649544:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04349 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.649558:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760702.649563:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04349 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.649572:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04349 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.649592:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04349 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.649599:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04349 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760702.650255:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493640192:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45491us (45750us total) trans 4294972480 rc 0/0 00000100:00100000:2.0:1789760702.650268:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5191 00000100:00100000:2.0:1789760702.650279:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712891422018 time 712945769785 state 0x2 ref 0 00000100:00100000:0.0:1789760702.653983:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760702.654175:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493640320 00000100:00100000:3.0:1789760702.654235:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493640320 00000100:00100000:2.0:1789760702.654464:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712891422018 time 712945769785 state 0x0 ref 1 00000100:00100000:2.0:1789760702.654482:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712946030201 time 712945769785 state 0x1 ref 1 00000100:00100000:2.0:1789760702.654567:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712946030201 time 712995601964 state 0x0 ref 1 00000100:00100000:2.0:1789760702.654581:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493640320:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760702.654634:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21143552, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:2.0:1789760702.654643:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04350 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.654659:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760702.654664:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760702.654670:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04350 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.654685:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04350 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760702.698716:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972481 is committed 00002000:00010000:2.0:1789760702.698935:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21143552 -> 21147648 00010000:00010000:2.0:1789760702.698951:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04350 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.698993:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04350 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.699003:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04350 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.699010:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760702.699015:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04350 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.699026:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04350 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.699048:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04350 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.699056:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04350 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760702.699154:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493640320:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44570us (45192us total) trans 4294972481 rc 0/0 00000100:00100000:2.0:1789760702.699164:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5192 00000100:00100000:2.0:1789760702.699173:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712946030201 time 712995601964 state 0x2 ref 0 00000100:00100000:2.0:1789760702.703544:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760702.704171:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493640448 00000100:00100000:1.0:1789760702.704181:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493640448 00000100:00100000:1.0:1789760702.704236:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712946030201 time 712995601964 state 0x0 ref 1 00000100:00100000:1.0:1789760702.704264:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712995862380 time 712995601964 state 0x1 ref 1 00000100:00100000:1.0:1789760702.704300:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712995862380 time 713045334939 state 0x0 ref 1 00000100:00100000:1.0:1789760702.704313:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493640448:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760702.704362:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21147648, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:1.0:1789760702.704371:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04357 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.704386:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760702.704391:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.704397:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04357 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.704411:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04357 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760702.742110:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972482 is committed 00002000:00010000:2.0:1789760702.742403:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21147648 -> 21151744 00010000:00010000:2.0:1789760702.742419:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04357 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.742433:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04357 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.742442:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04357 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.742449:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760702.742454:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04357 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.742470:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04357 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.742494:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04357 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.742503:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04357 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760702.742627:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493640448:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38311us (39088us total) trans 4294972482 rc 0/0 00000100:00100000:2.0:1789760702.742638:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5193 00000100:00100000:2.0:1789760702.742647:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712995862380 time 713045334939 state 0x2 ref 0 00000100:00100000:1.0:1789760702.745738:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760702.745843:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493640576 00000100:00100000:2.0:1789760702.745853:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493640576 00000100:00100000:2.0:1789760702.745908:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 712995862380 time 713045334939 state 0x0 ref 1 00000100:00100000:2.0:1789760702.745923:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713045595355 time 713045334939 state 0x1 ref 1 00000100:00100000:2.0:1789760702.745956:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713045595355 time 713086991303 state 0x0 ref 1 00000100:00100000:2.0:1789760702.745999:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493640576:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760702.746046:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21151744, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:2.0:1789760702.746053:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0435e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.746069:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760702.746074:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760702.746081:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0435e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.746104:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0435e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760702.803744:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972483 is committed 00002000:00010000:2.0:1789760702.803936:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21151744 -> 21155840 00010000:00010000:2.0:1789760702.803953:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0435e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.803995:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0435e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.804005:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0435e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.804013:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760702.804018:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0435e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.804028:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0435e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.804048:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0435e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.804058:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0435e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760702.804420:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493640576:12345-192.168.203.60@tcp:4:dd.0 Request processed in 58419us (58687us total) trans 4294972483 rc 0/0 00000100:00100000:0.0:1789760702.804430:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5194 00000100:00100000:0.0:1789760702.804440:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713045595355 time 713086991303 state 0x2 ref 0 00000100:00100000:0.0:1789760702.809853:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760702.810187:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493640704 00000100:00100000:3.0:1789760702.810198:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493640704 00000100:00100000:3.0:1789760702.810262:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713045595355 time 713086991303 state 0x0 ref 1 00000100:00100000:3.0:1789760702.810281:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713087251719 time 713086991303 state 0x1 ref 1 00000100:00100000:3.0:1789760702.810309:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713087251719 time 713151344629 state 0x0 ref 1 00000100:00100000:3.0:1789760702.810320:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493640704:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760702.810370:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21155840, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:3.0:1789760702.810379:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04365 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.810395:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760702.810399:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760702.810406:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04365 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760702.810419:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04365 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760702.845648:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc5837800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3305:1876699519245440:0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.845719:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760702.845782:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc5836300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519245952:0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.845807:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760702.845843:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789760702.845849:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc5836a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519245568:0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.845883:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760702.845922:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519245440 00000100:00100000:3.0:1789760702.845930:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519245440 00000100:00100000:3.0:1789760702.845991:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 444 00000100:00100000:3.0:1789760702.846001:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3305:x1876699519245440:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.846052:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3305:x1876699519245440:12345-0@lo:400:kworker.0 Request processed in 52us (334us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760702.846060:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 444 00000100:00100000:3.0:1789760702.846341:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc5837800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3305:1876699519245440:0@lo:400:kworker.0 00000100:00100000:1.0:1789760702.846420:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc5834e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3304:1876699519245312:0@lo:400:kworker.0 00000100:00100000:1.0:1789760702.846457:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760702.846490:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc5835500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3304:1876699519245824:0@lo:400:kworker.0 00000100:00100000:1.0:1789760702.846515:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760702.847211:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760702.847224:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc5835180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3304:1876699519245696:0@lo:400:kworker.0 00000100:00100000:1.0:1789760702.847317:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760702.847363:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519245952 00000100:00100000:1.0:1789760702.847369:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519245952 00000100:00100000:1.0:1789760702.847413:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519245568 00000100:00100000:1.0:1789760702.847417:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519245568 00000100:00100000:1.0:1789760702.847437:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 162 00000100:00100000:1.0:1789760702.847445:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519245952:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760702.847489:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519245952:12345-0@lo:400:kworker.0 Request processed in 44us (1679us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760702.847496:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 162 00000100:00100000:1.0:1789760702.847511:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 163 00000100:00100000:1.0:1789760702.847518:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519245568:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760702.847546:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519245568:12345-0@lo:400:kworker.0 Request processed in 30us (1663us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760702.847551:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 163 00000100:00100000:1.0:1789760702.847571:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519245824 00000100:00100000:1.0:1789760702.847575:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519245824 00000100:00100000:1.0:1789760702.847599:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519245696 00000100:00100000:1.0:1789760702.847602:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519245696 00000100:00100000:1.0:1789760702.847623:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 445 00000100:00100000:1.0:1789760702.847630:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3304:x1876699519245824:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760702.847666:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3304:x1876699519245824:12345-0@lo:400:kworker.0 Request processed in 37us (1150us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760702.847675:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 445 00000100:00100000:1.0:1789760702.847686:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 446 00000100:00100000:1.0:1789760702.847691:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3304:x1876699519245696:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760702.847718:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3304:x1876699519245696:12345-0@lo:400:kworker.0 Request processed in 27us (401us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760702.847724:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 446 00000100:00080000:1.0:1789760702.847742:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789760702.847746:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789760702.847787:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc5835180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3304:1876699519245696:0@lo:400:kworker.0 00000100:00100000:1.0:1789760702.847799:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc5835500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3304:1876699519245824:0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.848150:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519245312 00000100:00100000:3.0:1789760702.848156:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519245312 00000100:00100000:3.0:1789760702.852797:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 384 00000100:00100000:3.0:1789760702.852817:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d90c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3304:x1876699519245312:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760702.852927:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d90c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3304:x1876699519245312:12345-0@lo:400:kworker.0 Request processed in 117us (6468us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760702.852936:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 384 00000100:00100000:1.0:1789760702.853237:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc5836a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519245568:0@lo:400:kworker.0 00000100:00100000:1.0:1789760702.853260:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc5836300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519245952:0@lo:400:kworker.0 00000100:00100000:1.0:1789760702.853302:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc5834e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3304:1876699519245312:0@lo:400:kworker.0 00000001:00080000:0.0:1789760702.877039:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972484 is committed 00002000:00010000:2.0:1789760702.877447:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21155840 -> 21159936 00010000:00010000:2.0:1789760702.877464:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04365 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.877478:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04365 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.877488:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04365 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.877496:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760702.877501:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04365 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.877512:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04365 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.877534:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04365 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.877543:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04365 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760702.877696:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493640704:12345-192.168.203.60@tcp:4:dd.0 Request processed in 67373us (67849us total) trans 4294972484 rc 0/0 00000100:00100000:2.0:1789760702.877717:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5195 00000100:00100000:2.0:1789760702.877735:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713087251719 time 713151344629 state 0x2 ref 0 00000100:00100000:0.0:1789760702.889549:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760702.889655:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493640832 00000100:00100000:1.0:1789760702.889666:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493640832 00000100:00100000:1.0:1789760702.889937:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713087251719 time 713151344629 state 0x0 ref 1 00000100:00100000:1.0:1789760702.889953:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713151605045 time 713151344629 state 0x1 ref 1 00000100:00100000:1.0:1789760702.890005:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713151605045 time 713231040319 state 0x0 ref 1 00000100:00100000:1.0:1789760702.890018:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493640832:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760702.890066:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21159936, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:1.0:1789760702.890076:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0436c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.890608:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760702.890612:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.890618:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0436c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.890634:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0436c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760702.941602:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972485 is committed 00002000:00010000:2.0:1789760702.941948:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21159936 -> 21164032 00010000:00010000:2.0:1789760702.942001:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0436c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.942016:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0436c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.942025:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0436c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.942033:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760702.942039:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0436c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.942049:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0436c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.942072:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0436c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760702.942085:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0436c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760702.942179:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493640832:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52158us (52634us total) trans 4294972485 rc 0/0 00000100:00100000:2.0:1789760702.942190:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5196 00000100:00100000:2.0:1789760702.942201:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713151605045 time 713231040319 state 0x2 ref 0 00000100:00100000:0.0:1789760702.949937:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760702.950363:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493640960 00000100:00100000:1.0:1789760702.950374:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493640960 00000100:00100000:1.0:1789760702.950435:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713151605045 time 713231040319 state 0x0 ref 1 00000100:00100000:1.0:1789760702.950451:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713231300735 time 713231040319 state 0x1 ref 1 00000100:00100000:1.0:1789760702.950486:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713231300735 time 713291521924 state 0x0 ref 1 00000100:00100000:1.0:1789760702.950499:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493640960:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760702.950551:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21164032, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:1.0:1789760702.950559:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04373 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.950575:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760702.950581:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760702.950588:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04373 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760702.950602:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04373 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760703.004698:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972486 is committed 00002000:00010000:1.0:1789760703.005032:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21164032 -> 21168128 00010000:00010000:1.0:1789760703.005049:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04373 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.005061:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04373 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.005070:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04373 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.005077:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760703.005082:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04373 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.005090:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04373 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.005110:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04373 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.005118:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04373 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760703.005203:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493640960:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54700us (55479us total) trans 4294972486 rc 0/0 00000100:00100000:1.0:1789760703.005214:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5197 00000100:00100000:1.0:1789760703.005224:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713231300735 time 713291521924 state 0x2 ref 0 00000100:00100000:0.0:1789760703.009280:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760703.009377:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493641088 00000100:00100000:3.0:1789760703.009388:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493641088 00000100:00100000:3.0:1789760703.009452:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713231300735 time 713291521924 state 0x0 ref 1 00000100:00100000:3.0:1789760703.009467:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713291782340 time 713291521924 state 0x1 ref 1 00000100:00100000:3.0:1789760703.009494:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713291782340 time 713350530926 state 0x0 ref 1 00000100:00100000:3.0:1789760703.009506:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493641088:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760703.009554:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21168128, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:3.0:1789760703.009562:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0437a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.009583:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760703.009588:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760703.009595:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0437a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.009609:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0437a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760703.010338:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789760703.045228:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760703.115604:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972487 is committed 00002000:00010000:2.0:1789760703.116546:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21168128 -> 21172224 00010000:00010000:2.0:1789760703.116563:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0437a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.116576:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0437a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.116584:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0437a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.116592:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760703.116597:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0437a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.116606:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0437a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.116630:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0437a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.116638:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0437a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760703.116868:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493641088:12345-192.168.203.60@tcp:4:dd.0 Request processed in 107357us (107592us total) trans 4294972487 rc 0/0 00000100:00100000:2.0:1789760703.116879:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5198 00000100:00100000:2.0:1789760703.116890:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713291782340 time 713350530926 state 0x2 ref 0 00000100:00100000:0.0:1789760703.121225:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760703.121296:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493641216 00000100:00100000:3.0:1789760703.121306:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493641216 00000100:00100000:3.0:1789760703.121356:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713291782340 time 713350530926 state 0x0 ref 1 00000100:00100000:3.0:1789760703.121370:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713350791342 time 713350530926 state 0x1 ref 1 00000100:00100000:3.0:1789760703.121402:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713350791342 time 713462437840 state 0x0 ref 1 00000100:00100000:3.0:1789760703.121414:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493641216:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760703.121463:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21172224, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:3.0:1789760703.121472:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04381 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.121487:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760703.121491:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760703.121497:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04381 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.121515:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04381 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760703.187869:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21172224 -> 21176320 00010000:00010000:2.0:1789760703.187890:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04381 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.187904:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04381 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.187913:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04381 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.187921:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760703.187926:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04381 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.187942:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04381 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.187982:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04381 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.187992:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04381 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760703.188093:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493641216:12345-192.168.203.60@tcp:4:dd.0 Request processed in 66676us (66873us total) trans 4294972488 rc 0/0 00000100:00100000:2.0:1789760703.188104:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5199 00000100:00100000:2.0:1789760703.188115:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713350791342 time 713462437840 state 0x2 ref 0 00000001:00080000:1.0:1789760703.189743:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972488 is committed 00000100:00100000:0.0:1789760703.192520:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760703.192654:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493641344 00000100:00100000:1.0:1789760703.192665:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493641344 00000100:00100000:1.0:1789760703.192717:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713350791342 time 713462437840 state 0x0 ref 1 00000100:00100000:1.0:1789760703.192731:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713462698256 time 713462437840 state 0x1 ref 1 00000100:00100000:1.0:1789760703.192798:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713462698256 time 713533832938 state 0x0 ref 1 00000100:00100000:1.0:1789760703.192811:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493641344:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760703.192925:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21176320, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:1.0:1789760703.192934:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04388 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.192950:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760703.192954:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760703.192980:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04388 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.193000:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04388 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760703.237357:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21176320 -> 21180416 00010000:00010000:2.0:1789760703.237378:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04388 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.237391:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04388 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.237401:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04388 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.237409:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760703.237415:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04388 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.237425:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04388 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.237444:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04388 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.237452:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04388 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760703.237545:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493641344:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44733us (45030us total) trans 4294972489 rc 0/0 00000100:00100000:2.0:1789760703.237556:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5200 00000100:00100000:2.0:1789760703.237565:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713462698256 time 713533832938 state 0x2 ref 0 00000001:00080000:3.0:1789760703.240337:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972489 is committed 00000100:00100000:0.0:1789760703.240996:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760703.241109:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493641472 00000100:00100000:1.0:1789760703.241118:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493641472 00000100:00100000:1.0:1789760703.241171:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713462698256 time 713533832938 state 0x0 ref 1 00000100:00100000:1.0:1789760703.241185:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713534093354 time 713533832938 state 0x1 ref 1 00000100:00100000:1.0:1789760703.241215:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713534093354 time 713582250262 state 0x0 ref 1 00000100:00100000:1.0:1789760703.241227:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493641472:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760703.241285:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21180416, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:1.0:1789760703.241295:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0438f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.241311:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760703.241315:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760703.241321:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0438f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.241334:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0438f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760703.300455:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972490 is committed 00002000:00010000:3.0:1789760703.300521:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21180416 -> 21184512 00010000:00010000:3.0:1789760703.300536:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0438f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.300550:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0438f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.300559:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0438f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.300566:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760703.300572:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0438f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.300580:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0438f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.300600:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0438f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.300607:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0438f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760703.300709:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493641472:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59480us (59713us total) trans 4294972490 rc 0/0 00000100:00100000:3.0:1789760703.300720:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5201 00000100:00100000:3.0:1789760703.300730:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713534093354 time 713582250262 state 0x2 ref 0 00000100:00100000:0.0:1789760703.305153:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760703.305438:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493641600 00000100:00100000:2.0:1789760703.305449:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493641600 00000100:00100000:2.0:1789760703.305508:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713534093354 time 713582250262 state 0x0 ref 1 00000100:00100000:2.0:1789760703.305524:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713582510678 time 713582250262 state 0x1 ref 1 00000100:00100000:2.0:1789760703.305560:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713582510678 time 713646595747 state 0x0 ref 1 00000100:00100000:2.0:1789760703.305574:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493641600:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760703.305625:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21184512, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:2.0:1789760703.305634:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04396 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.305662:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760703.305667:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760703.305672:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04396 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.305686:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04396 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760703.356579:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789760703.356602:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc2884700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519246080:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760703.357482:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760703.357914:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519246080 00000100:00100000:1.0:1789760703.357925:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519246080 00000100:00100000:1.0:1789760703.358911:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 185 00000100:00100000:1.0:1789760703.358928:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519246080:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760703.358983:0:6690:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760703.359055:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519246080:12345-0@lo:13:osp-pre-1-0.0 Request processed in 132us (2233us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760703.359062:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 185 00000100:00100000:2.0:1789760703.359289:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc2884700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519246080:0@lo:13:osp-pre-1-0.0 00000001:00080000:3.0:1789760703.390360:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972491 is committed 00002000:00010000:1.0:1789760703.390603:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21184512 -> 21188608 00010000:00010000:1.0:1789760703.390619:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04396 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.390633:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04396 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.390656:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04396 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.390664:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760703.390669:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04396 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.390681:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04396 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.390700:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04396 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.390713:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04396 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760703.390809:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493641600:12345-192.168.203.60@tcp:4:dd.0 Request processed in 85232us (85660us total) trans 4294972491 rc 0/0 00000100:00100000:1.0:1789760703.390826:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5202 00000100:00100000:1.0:1789760703.390836:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713582510678 time 713646595747 state 0x2 ref 0 00000100:00100000:1.0:1789760703.404198:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760703.404306:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493641728 00000100:00100000:2.0:1789760703.404315:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493641728 00000100:00100000:2.0:1789760703.404372:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713582510678 time 713646595747 state 0x0 ref 1 00000100:00100000:2.0:1789760703.404387:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713646856163 time 713646595747 state 0x1 ref 1 00000100:00100000:2.0:1789760703.404420:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713646856163 time 713745455385 state 0x0 ref 1 00000100:00100000:2.0:1789760703.404433:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493641728:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760703.404482:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21188608, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:2.0:1789760703.404491:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0439d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.404508:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760703.404513:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760703.404520:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0439d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.404533:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0439d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760703.491221:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972492 is committed 00002000:00010000:1.0:1789760703.491469:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21188608 -> 21192704 00010000:00010000:1.0:1789760703.491485:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0439d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.491499:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0439d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.491509:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0439d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.491517:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760703.491522:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0439d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.491532:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0439d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.491557:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0439d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.491564:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0439d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760703.491780:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493641728:12345-192.168.203.60@tcp:4:dd.0 Request processed in 87344us (87587us total) trans 4294972492 rc 0/0 00000100:00100000:1.0:1789760703.491800:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5203 00000100:00100000:1.0:1789760703.491810:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713646856163 time 713745455385 state 0x2 ref 0 00000100:00100000:3.0:1789760703.496987:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760703.497055:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493641856 00000100:00100000:1.0:1789760703.497064:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493641856 00000100:00100000:1.0:1789760703.497120:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713646856163 time 713745455385 state 0x0 ref 1 00000100:00100000:1.0:1789760703.497135:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713745715801 time 713745455385 state 0x1 ref 1 00000100:00100000:1.0:1789760703.497166:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713745715801 time 713838201325 state 0x0 ref 1 00000100:00100000:1.0:1789760703.497179:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493641856:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760703.497229:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21192704, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:1.0:1789760703.497238:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043a4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.497266:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760703.497272:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760703.497278:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.497293:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760703.528230:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972493 is committed 00002000:00010000:1.0:1789760703.528570:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21192704 -> 21196800 00010000:00010000:1.0:1789760703.528588:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.528602:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043a4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.528611:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.528618:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760703.528623:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.528633:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043a4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.528653:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043a4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.528660:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043a4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760703.528808:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493641856:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31625us (31846us total) trans 4294972493 rc 0/0 00000100:00100000:1.0:1789760703.528829:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5204 00000100:00100000:1.0:1789760703.528839:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713745715801 time 713838201325 state 0x2 ref 0 00000100:00100000:0.0:1789760703.535909:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760703.536044:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493641984 00000100:00100000:3.0:1789760703.536055:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493641984 00000100:00100000:3.0:1789760703.536117:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713745715801 time 713838201325 state 0x0 ref 1 00000100:00100000:3.0:1789760703.536132:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713838461741 time 713838201325 state 0x1 ref 1 00000100:00100000:3.0:1789760703.536162:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713838461741 time 713877197402 state 0x0 ref 1 00000100:00100000:3.0:1789760703.536174:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493641984:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760703.536220:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21196800, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:3.0:1789760703.536229:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e043ab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.536244:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760703.536249:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760703.536255:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e043ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.536269:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e043ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760703.594254:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972494 is committed 00002000:00010000:2.0:1789760703.594595:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21196800 -> 21200896 00010000:00010000:2.0:1789760703.594611:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e043ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.594625:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e043ab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.594634:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e043ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.594641:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760703.594645:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e043ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.594655:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e043ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.594675:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e043ab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.594684:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e043ab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760703.598640:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760703.598774:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493641984:12345-192.168.203.60@tcp:4:dd.0 Request processed in 58606us (58879us total) trans 4294972494 rc 0/0 00000100:00100000:2.0:1789760703.598801:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5205 00000100:00100000:2.0:1789760703.598954:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713838461741 time 713877197402 state 0x2 ref 0 00000100:00100000:0.0:1789760703.599063:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493642112 00000100:00100000:0.0:1789760703.599072:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493642112 00000100:00100000:0.0:1789760703.599118:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713838461741 time 713877197402 state 0x0 ref 1 00000100:00100000:0.0:1789760703.599133:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713877457818 time 713877197402 state 0x1 ref 1 00000100:00100000:0.0:1789760703.599162:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713877457818 time 713940197365 state 0x0 ref 1 00000100:00100000:0.0:1789760703.599173:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493642112:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760703.599223:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21200896, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0xa807 00010000:00010000:0.0:1789760703.599231:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e043b2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.599257:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760703.599261:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760703.599266:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e043b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.599279:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e043b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760703.661999:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972495 is committed 00002000:00010000:2.0:1789760703.662653:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21200896 -> 21204992 00002000:00010000:2.0:1789760703.662658:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760702 -> 1789760703 00002000:00010000:2.0:1789760703.662661:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760702 -> 1789760703 00010000:00010000:2.0:1789760703.662675:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e043b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.662689:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e043b2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.662699:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e043b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.662707:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760703.662712:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e043b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.662728:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e043b2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.662748:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e043b2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.662757:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e043b2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760703.663006:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493642112:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63828us (64368us total) trans 4294972495 rc 0/0 00000100:00100000:2.0:1789760703.663017:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5206 00000100:00100000:2.0:1789760703.663027:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713877457818 time 713940197365 state 0x2 ref 0 00000100:00100000:2.0:1789760703.669435:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760703.669540:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493642240 00000100:00100000:1.0:1789760703.669551:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493642240 00000100:00100000:1.0:1789760703.669617:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713877457818 time 713940197365 state 0x0 ref 1 00000100:00100000:1.0:1789760703.669635:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713940457781 time 713940197365 state 0x1 ref 1 00000100:00100000:1.0:1789760703.669697:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713940457781 time 714010731754 state 0x0 ref 1 00000100:00100000:1.0:1789760703.669711:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493642240:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760703.669776:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21204992, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa807 00010000:00010000:1.0:1789760703.669787:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043b9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.669804:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760703.669809:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760703.669886:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.669902:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760703.723312:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972496 is committed 00002000:00010000:1.0:1789760703.723401:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21204992 -> 21209088 00010000:00010000:1.0:1789760703.723418:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.723432:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043b9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.723441:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.723449:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760703.723453:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.723463:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043b9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.723482:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043b9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.723490:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043b9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760703.723625:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493642240:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53911us (54194us total) trans 4294972496 rc 0/0 00000100:00100000:1.0:1789760703.723639:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5207 00000100:00100000:1.0:1789760703.723650:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713940457781 time 714010731754 state 0x2 ref 0 00000100:00100000:0.0:1789760703.735136:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760703.741810:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493642368 00000100:00100000:1.0:1789760703.741826:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493642368 00000100:00100000:1.0:1789760703.741985:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 713940457781 time 714010731754 state 0x0 ref 1 00000100:00100000:1.0:1789760703.742001:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714010992170 time 714010731754 state 0x1 ref 1 00000100:00100000:1.0:1789760703.742099:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714010992170 time 714083132806 state 0x0 ref 1 00000100:00100000:1.0:1789760703.742115:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493642368:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760703.742180:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21209088, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa807 00010000:00010000:1.0:1789760703.742190:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043c0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.742206:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760703.742211:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760703.742217:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.742232:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760703.813993:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972497 is committed 00002000:00010000:2.0:1789760703.814146:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21209088 -> 21213184 00010000:00010000:2.0:1789760703.814165:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.814179:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043c0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.814200:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.814213:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760703.814218:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.814226:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043c0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.814254:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043c0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.814263:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e043c0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760703.814641:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493642368:12345-192.168.203.60@tcp:4:dd.0 Request processed in 72525us (79510us total) trans 4294972497 rc 0/0 00000100:00100000:2.0:1789760703.814653:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5208 00000100:00100000:2.0:1789760703.814664:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714010992170 time 714083132806 state 0x2 ref 0 00000100:00100000:1.0:1789760703.831085:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760703.831182:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493642496 00000100:00100000:2.0:1789760703.831193:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493642496 00000100:00100000:2.0:1789760703.831261:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714010992170 time 714083132806 state 0x0 ref 1 00000100:00100000:2.0:1789760703.831278:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714083393222 time 714083132806 state 0x1 ref 1 00000100:00100000:2.0:1789760703.831309:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714083393222 time 714172344668 state 0x0 ref 1 00000100:00100000:2.0:1789760703.831324:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493642496:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760703.831384:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21213184, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa807 00010000:00010000:2.0:1789760703.831394:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e043c7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.831411:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760703.831415:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760703.831422:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e043c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.831437:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e043c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760703.869180:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972498 is committed 00002000:00010000:2.0:1789760703.869276:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21213184 -> 21217280 00010000:00010000:2.0:1789760703.869294:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e043c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.869309:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e043c7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.869318:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e043c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.869326:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760703.869330:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e043c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.869340:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e043c7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.869358:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e043c7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.869366:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e043c7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760703.869451:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493642496:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38125us (38372us total) trans 4294972498 rc 0/0 00000100:00100000:2.0:1789760703.869461:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5209 00000100:00100000:2.0:1789760703.869471:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714083393222 time 714172344668 state 0x2 ref 0 00000100:00100000:3.0:1789760703.873067:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760703.873176:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493642624 00000100:00100000:2.0:1789760703.873187:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493642624 00000100:00100000:2.0:1789760703.873250:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714083393222 time 714172344668 state 0x0 ref 1 00000100:00100000:2.0:1789760703.873266:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714172605084 time 714172344668 state 0x1 ref 1 00000100:00100000:2.0:1789760703.873296:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714172605084 time 714214331823 state 0x0 ref 1 00000100:00100000:2.0:1789760703.873307:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493642624:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760703.873354:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21217280, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa807 00010000:00010000:2.0:1789760703.873364:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e043ce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.873379:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760703.873384:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760703.873390:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e043ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.873405:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e043ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760703.905615:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21217280 -> 21221376 00010000:00010000:3.0:1789760703.905637:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e043ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.905652:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e043ce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.905673:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e043ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.905680:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760703.905685:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e043ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.905694:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e043ce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.905714:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e043ce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760703.905727:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e043ce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760703.905941:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493642624:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32628us (32877us total) trans 4294972499 rc 0/0 00000100:00100000:3.0:1789760703.905952:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5210 00000100:00100000:3.0:1789760703.905999:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714172605084 time 714214331823 state 0x2 ref 0 00000001:00080000:1.0:1789760703.907028:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972499 is committed 00000100:00100000:3.0:1789760703.909136:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760703.909873:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493642752 00000100:00100000:1.0:1789760703.909883:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493642752 00000100:00100000:1.0:1789760703.909940:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714172605084 time 714214331823 state 0x0 ref 1 00000100:00100000:1.0:1789760703.909955:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714214592239 time 714214331823 state 0x1 ref 1 00000100:00100000:2.0:1789760703.910301:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714214592239 time 714251333651 state 0x0 ref 1 00000100:00100000:2.0:1789760703.910318:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493642752:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760703.910363:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21221376, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa807 00010000:00010000:2.0:1789760703.910372:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e043d5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.910386:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760703.910390:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760703.910396:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e043d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.910409:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e043d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760703.944835:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972500 is committed 00002000:00010000:0.0:1789760703.944953:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21221376 -> 21225472 00010000:00010000:0.0:1789760703.945000:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e043d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.945015:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e043d5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.945028:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e043d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.945035:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760703.945040:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e043d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.945049:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e043d5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.945068:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e043d5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.945074:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e043d5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760703.945942:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493642752:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35621us (36806us total) trans 4294972500 rc 0/0 00000100:00100000:1.0:1789760703.945955:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5211 00000100:00100000:1.0:1789760703.945986:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714214592239 time 714251333651 state 0x2 ref 0 00000100:00100000:0.0:1789760703.950824:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760703.950952:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493642880 00000100:00100000:1.0:1789760703.951005:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493642880 00000100:00100000:1.0:1789760703.951059:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714214592239 time 714251333651 state 0x0 ref 1 00000100:00100000:1.0:1789760703.951074:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714251594067 time 714251333651 state 0x1 ref 1 00000100:00100000:1.0:1789760703.951160:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714251594067 time 714292186320 state 0x0 ref 1 00000100:00100000:1.0:1789760703.951173:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493642880:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760703.951224:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21225472, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa807 00010000:00010000:1.0:1789760703.951252:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e043dc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.951267:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760703.951272:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760703.951279:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e043dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.951294:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e043dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760703.980842:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21225472 -> 21229568 00010000:00010000:1.0:1789760703.980862:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e043dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.980876:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e043dc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.980885:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e043dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.980892:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760703.980897:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e043dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.980906:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e043dc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.980924:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e043dc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760703.980933:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e043dc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760703.981053:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493642880:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29877us (30244us total) trans 4294972501 rc 0/0 00000100:00100000:1.0:1789760703.981062:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5212 00000100:00100000:1.0:1789760703.981071:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714251594067 time 714292186320 state 0x2 ref 0 00000001:00080000:1.0:1789760703.981106:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972501 is committed 00000100:00100000:3.0:1789760703.986355:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760703.987189:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493643008 00000100:00100000:2.0:1789760703.987203:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493643008 00000100:00100000:2.0:1789760703.987265:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714251594067 time 714292186320 state 0x0 ref 1 00000100:00100000:2.0:1789760703.987283:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714292446736 time 714292186320 state 0x1 ref 1 00000100:00100000:2.0:1789760703.987318:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714292446736 time 714328353276 state 0x0 ref 1 00000100:00100000:2.0:1789760703.987331:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9621016d2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493643008:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760703.987382:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21229568, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa807 00010000:00010000:2.0:1789760703.987393:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e043e3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.987409:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760703.987414:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760703.987421:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e043e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760703.987436:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e043e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760704.023518:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972502 is committed 00002000:00010000:2.0:1789760704.023602:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21229568 -> 21233664 00010000:00010000:2.0:1789760704.023619:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e043e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.023631:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e043e3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.023640:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e043e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.023648:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760704.023653:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e043e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.023662:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e043e3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.023682:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e043e3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.023690:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e043e3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.025346:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9621016d2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493643008:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38006us (38989us total) trans 4294972502 rc 0/0 00000100:00100000:2.0:1789760704.025361:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5213 00000100:00100000:2.0:1789760704.025371:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714292446736 time 714328353276 state 0x2 ref 0 00000100:00100000:3.0:1789760704.031737:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760704.032703:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493643136 00000100:00100000:2.0:1789760704.032715:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493643136 00000100:00100000:3.0:1789760704.032941:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714292446736 time 714328353276 state 0x0 ref 1 00000100:00100000:3.0:1789760704.032957:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714328613692 time 714328353276 state 0x1 ref 1 00000100:00100000:3.0:1789760704.033058:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714328613692 time 714374092548 state 0x0 ref 1 00000100:00100000:3.0:1789760704.033073:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493643136:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760704.033122:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21233664, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa807 00010000:00010000:3.0:1789760704.033132:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e043ea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.033148:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760704.033152:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760704.033160:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e043ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.033175:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e043ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.058196:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760704.069727:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21233664 -> 21237760 00010000:00010000:1.0:1789760704.069745:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e043ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.069770:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e043ea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.069780:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e043ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.069787:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760704.069792:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e043ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.069801:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e043ea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.069821:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e043ea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760704.069827:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972503 is committed 00010000:00010000:1.0:1789760704.069830:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e043ea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760704.070016:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493643136:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36942us (38297us total) trans 4294972503 rc 0/0 00000100:00100000:1.0:1789760704.070027:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5214 00000100:00100000:1.0:1789760704.070037:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714328613692 time 714374092548 state 0x2 ref 0 00000100:00100000:1.0:1789760704.073669:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760704.073794:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493643264 00000100:00100000:2.0:1789760704.073804:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493643264 00000100:00100000:2.0:1789760704.073861:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714328613692 time 714374092548 state 0x0 ref 1 00000100:00100000:2.0:1789760704.073876:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714374352964 time 714374092548 state 0x1 ref 1 00000100:00100000:2.0:1789760704.073911:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714374352964 time 714414946111 state 0x0 ref 1 00000100:00100000:2.0:1789760704.073924:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493643264:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760704.074006:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21237760, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa807 00010000:00010000:2.0:1789760704.074016:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e043f1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.074032:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760704.074036:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.074043:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e043f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.074059:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e043f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760704.115088:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21237760 -> 21241856 00010000:00010000:2.0:1789760704.115110:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e043f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.115124:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e043f1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.115133:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e043f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.115141:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760704.115147:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e043f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.115157:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e043f1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.115179:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e043f1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.115187:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e043f1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.115300:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493643264:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41371us (41633us total) trans 4294972504 rc 0/0 00000100:00100000:2.0:1789760704.115311:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5215 00000100:00100000:2.0:1789760704.115321:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714374352964 time 714414946111 state 0x2 ref 0 00000001:00080000:1.0:1789760704.117635:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972504 is committed 00000100:00100000:3.0:1789760704.126370:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760704.126597:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493643392 00000100:00100000:1.0:1789760704.126608:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493643392 00000100:00100000:1.0:1789760704.126671:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714374352964 time 714414946111 state 0x0 ref 1 00000100:00100000:1.0:1789760704.126687:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714415206527 time 714414946111 state 0x1 ref 1 00000100:00100000:1.0:1789760704.126720:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714415206527 time 714467754974 state 0x0 ref 1 00000100:00100000:1.0:1789760704.126732:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493643392:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760704.126795:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21241856, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa807 00010000:00010000:1.0:1789760704.126805:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e043f8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.126830:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760704.126835:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760704.126842:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e043f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.126857:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e043f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760704.170269:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972505 is committed 00002000:00010000:2.0:1789760704.170430:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21241856 -> 21245952 00010000:00010000:2.0:1789760704.170446:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e043f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.170459:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e043f8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.170468:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e043f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.170481:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760704.170486:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e043f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.170495:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e043f8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.170515:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e043f8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.170524:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e043f8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.170615:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493643392:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43880us (44250us total) trans 4294972505 rc 0/0 00000100:00100000:2.0:1789760704.170626:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5216 00000100:00100000:2.0:1789760704.170637:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714415206527 time 714467754974 state 0x2 ref 0 00000100:00100000:3.0:1789760704.177068:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760704.177244:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493643520 00000100:00100000:0.0:1789760704.177255:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493643520 00000100:00100000:0.0:1789760704.177310:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714415206527 time 714467754974 state 0x0 ref 1 00000100:00100000:0.0:1789760704.177326:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714468015390 time 714467754974 state 0x1 ref 1 00000100:00100000:0.0:1789760704.177358:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714468015390 time 714518392750 state 0x0 ref 1 00000100:00100000:0.0:1789760704.177373:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493643520:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760704.177421:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21245952, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa807 00010000:00010000:0.0:1789760704.177430:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e043ff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.177444:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760704.177448:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760704.177453:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e043ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.177466:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e043ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760704.198125:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972506 is committed 00002000:00010000:2.0:1789760704.198254:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21245952 -> 21250048 00010000:00010000:2.0:1789760704.198270:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e043ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.198283:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e043ff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.198297:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e043ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.198304:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760704.198309:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e043ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.198318:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e043ff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.198342:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e043ff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.198352:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e043ff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.198449:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493643520:12345-192.168.203.60@tcp:4:dd.0 Request processed in 21074us (21386us total) trans 4294972506 rc 0/0 00000100:00100000:2.0:1789760704.198460:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5217 00000100:00100000:2.0:1789760704.198470:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714468015390 time 714518392750 state 0x2 ref 0 00000100:00100000:0.0:1789760704.201287:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760704.201410:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493643648 00000100:00100000:3.0:1789760704.201420:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493643648 00000100:00100000:3.0:1789760704.201470:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714468015390 time 714518392750 state 0x0 ref 1 00000100:00100000:3.0:1789760704.201485:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714518653166 time 714518392750 state 0x1 ref 1 00000100:00100000:3.0:1789760704.201519:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714518653166 time 714542554828 state 0x0 ref 1 00000100:00100000:3.0:1789760704.201529:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493643648:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760704.201567:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21250048, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa807 00010000:00010000:3.0:1789760704.201573:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04406 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.201585:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760704.201589:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760704.201593:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04406 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.201607:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04406 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760704.230718:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21250048 -> 21254144 00010000:00010000:3.0:1789760704.230741:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04406 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.230766:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04406 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.230776:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04406 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.230784:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760704.230789:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04406 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.230798:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04406 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.230876:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04406 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.230886:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04406 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760704.231067:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972507 is committed 00000100:00100000:3.0:1789760704.231802:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493643648:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30268us (30520us total) trans 4294972507 rc 0/0 00000100:00100000:3.0:1789760704.231822:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5218 00000100:00100000:3.0:1789760704.231831:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714518653166 time 714542554828 state 0x2 ref 0 00000100:00100000:3.0:1789760704.234589:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760704.234735:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493643776 00000100:00100000:2.0:1789760704.234745:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493643776 00000100:00100000:2.0:1789760704.234818:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714518653166 time 714542554828 state 0x0 ref 1 00000100:00100000:2.0:1789760704.234838:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714542815244 time 714542554828 state 0x1 ref 1 00000100:00100000:2.0:1789760704.234872:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714542815244 time 714575907324 state 0x0 ref 1 00000100:00100000:2.0:1789760704.234885:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493643776:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760704.234930:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21254144, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa807 00010000:00010000:2.0:1789760704.234939:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0440d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.234994:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760704.235002:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.235008:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0440d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.235022:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0440d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760704.270496:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972508 is committed 00002000:00010000:1.0:1789760704.270595:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21254144 -> 21258240 00010000:00010000:1.0:1789760704.270611:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0440d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.270624:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0440d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.270633:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0440d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.270641:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760704.270646:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0440d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.270656:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0440d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.270676:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0440d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.270683:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0440d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760704.270831:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493643776:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35942us (36245us total) trans 4294972508 rc 0/0 00000100:00100000:1.0:1789760704.270841:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5219 00000100:00100000:1.0:1789760704.270852:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714542815244 time 714575907324 state 0x2 ref 0 00000100:00100000:0.0:1789760704.276842:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760704.276977:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493643904 00000100:00100000:2.0:1789760704.276987:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493643904 00000100:00100000:2.0:1789760704.277048:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714542815244 time 714575907324 state 0x0 ref 1 00000100:00100000:2.0:1789760704.277063:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714576167740 time 714575907324 state 0x1 ref 1 00000100:00100000:2.0:1789760704.277089:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714576167740 time 714618124169 state 0x0 ref 1 00000100:00100000:2.0:1789760704.277101:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493643904:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760704.277152:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21258240, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa807 00010000:00010000:2.0:1789760704.277162:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04414 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.277180:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760704.277185:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.277191:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04414 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.277206:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04414 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760704.314709:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21258240 -> 21262336 00010000:00010000:0.0:1789760704.314728:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04414 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.314741:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04414 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.314751:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04414 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.314775:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760704.314780:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04414 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.314789:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04414 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.314809:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04414 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.314816:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04414 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760704.314909:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493643904:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37805us (38072us total) trans 4294972509 rc 0/0 00000100:00100000:0.0:1789760704.314919:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5220 00000100:00100000:0.0:1789760704.314929:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714576167740 time 714618124169 state 0x2 ref 0 00000001:00080000:3.0:1789760704.315826:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972509 is committed 00000100:00100000:0.0:1789760704.319940:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760704.320390:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493644032 00000100:00100000:2.0:1789760704.320399:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493644032 00000100:00100000:2.0:1789760704.320449:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714576167740 time 714618124169 state 0x0 ref 1 00000100:00100000:2.0:1789760704.320463:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714618384585 time 714618124169 state 0x1 ref 1 00000100:00100000:2.0:1789760704.320491:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714618384585 time 714661526950 state 0x0 ref 1 00000100:00100000:2.0:1789760704.320502:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493644032:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760704.320547:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21262336, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa807 00010000:00010000:2.0:1789760704.320556:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0441b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.320571:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760704.320577:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.320583:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0441b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.320596:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0441b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760704.371611:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21262336 -> 21266432 00010000:00010000:0.0:1789760704.371632:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0441b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.371644:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0441b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.371659:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0441b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.371666:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760704.371671:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0441b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.371680:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0441b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.371699:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0441b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.371707:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0441b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760704.371815:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493644032:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51302us (51871us total) trans 4294972510 rc 0/0 00000100:00100000:0.0:1789760704.371826:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5221 00000100:00100000:0.0:1789760704.371835:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714618384585 time 714661526950 state 0x2 ref 0 00000001:00080000:3.0:1789760704.372017:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972510 is committed 00000100:00100000:1.0:1789760704.381185:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760704.381275:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493644160 00000100:00100000:3.0:1789760704.381285:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493644160 00000100:00100000:3.0:1789760704.381335:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714618384585 time 714661526950 state 0x0 ref 1 00000100:00100000:3.0:1789760704.381349:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714661787366 time 714661526950 state 0x1 ref 1 00000100:00100000:3.0:1789760704.381379:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714661787366 time 714722415215 state 0x0 ref 1 00000100:00100000:3.0:1789760704.381390:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493644160:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760704.381437:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21266432, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa807 00010000:00010000:3.0:1789760704.381446:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04422 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.381461:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760704.381466:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760704.381472:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04422 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.381490:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04422 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760704.443621:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972511 is committed 00002000:00010000:2.0:1789760704.444575:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21266432 -> 21270528 00010000:00010000:2.0:1789760704.444591:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04422 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.444604:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04422 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.444613:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04422 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.444620:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760704.444626:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04422 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.444636:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04422 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.444664:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04422 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.444671:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04422 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.444773:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493644160:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63379us (63595us total) trans 4294972511 rc 0/0 00000100:00100000:2.0:1789760704.444789:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5222 00000100:00100000:2.0:1789760704.444799:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714661787366 time 714722415215 state 0x2 ref 0 00000100:00100000:1.0:1789760704.453107:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760704.453222:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493644288 00000100:00100000:2.0:1789760704.453232:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493644288 00000100:00100000:2.0:1789760704.453287:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714661787366 time 714722415215 state 0x0 ref 1 00000100:00100000:2.0:1789760704.453304:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714722675631 time 714722415215 state 0x1 ref 1 00000100:00100000:2.0:1789760704.453337:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714722675631 time 714794372279 state 0x0 ref 1 00000100:00100000:2.0:1789760704.453350:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493644288:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760704.453399:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21270528, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa807 00010000:00010000:2.0:1789760704.453408:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04429 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.453423:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760704.453428:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.453434:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04429 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.453449:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04429 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760704.527629:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972512 is committed 00002000:00010000:2.0:1789760704.529666:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21270528 -> 21274624 00010000:00010000:2.0:1789760704.529685:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04429 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.529698:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04429 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.529707:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04429 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.529715:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760704.529720:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04429 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.529729:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04429 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.529747:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04429 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.529766:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04429 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.529873:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493644288:12345-192.168.203.60@tcp:4:dd.0 Request processed in 76520us (76772us total) trans 4294972512 rc 0/0 00000100:00100000:2.0:1789760704.529887:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5223 00000100:00100000:2.0:1789760704.529896:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714722675631 time 714794372279 state 0x2 ref 0 00000100:00100000:0.0:1789760704.534498:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760704.534575:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493644416 00000100:00100000:2.0:1789760704.534585:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493644416 00000100:00100000:2.0:1789760704.534640:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714722675631 time 714794372279 state 0x0 ref 1 00000100:00100000:2.0:1789760704.534655:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714794632695 time 714794372279 state 0x1 ref 1 00000100:00100000:2.0:1789760704.534688:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714794632695 time 714875723457 state 0x0 ref 1 00000100:00100000:2.0:1789760704.534700:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493644416:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760704.534750:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21274624, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa807 00010000:00010000:2.0:1789760704.534767:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04430 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.534784:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760704.534789:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.534796:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04430 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.534818:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04430 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760704.621671:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972513 is committed 00002000:00010000:2.0:1789760704.621787:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21274624 -> 21278720 00010000:00010000:2.0:1789760704.621802:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04430 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.621825:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04430 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.621835:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04430 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.621842:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760704.621847:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04430 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.621857:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04430 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.621875:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04430 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.621883:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04430 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.622018:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493644416:12345-192.168.203.60@tcp:4:dd.0 Request processed in 87315us (87526us total) trans 4294972513 rc 0/0 00000100:00100000:2.0:1789760704.622029:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5224 00000100:00100000:2.0:1789760704.622040:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714794632695 time 714875723457 state 0x2 ref 0 00000100:00100000:1.0:1789760704.625312:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760704.625416:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493644544 00000100:00100000:0.0:1789760704.625426:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493644544 00000100:00100000:0.0:1789760704.625481:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714794632695 time 714875723457 state 0x0 ref 1 00000100:00100000:0.0:1789760704.625502:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714875983873 time 714875723457 state 0x1 ref 1 00000100:00100000:0.0:1789760704.625536:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714875983873 time 714966571821 state 0x0 ref 1 00000100:00100000:0.0:1789760704.625550:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493644544:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760704.625606:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21278720, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0xa807 00010000:00010000:0.0:1789760704.625617:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04437 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.625634:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760704.625639:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760704.625645:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04437 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.625659:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04437 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760704.711733:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972514 is committed 00002000:00010000:3.0:1789760704.712210:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21278720 -> 21282816 00002000:00010000:3.0:1789760704.712215:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760703 -> 1789760704 00002000:00010000:3.0:1789760704.712218:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760703 -> 1789760704 00010000:00010000:3.0:1789760704.712231:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04437 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.712243:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04437 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.712252:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04437 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.712260:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760704.712265:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04437 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.712274:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04437 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.712295:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04437 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.712304:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04437 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760704.712396:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493644544:12345-192.168.203.60@tcp:4:dd.0 Request processed in 86844us (87088us total) trans 4294972514 rc 0/0 00000100:00100000:3.0:1789760704.712406:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5225 00000100:00100000:3.0:1789760704.712417:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714875983873 time 714966571821 state 0x2 ref 0 00000100:00100000:1.0:1789760704.728230:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760704.728443:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493644672 00000100:00100000:0.0:1789760704.728454:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493644672 00000100:00100000:0.0:1789760704.728509:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714875983873 time 714966571821 state 0x0 ref 1 00000100:00100000:0.0:1789760704.728526:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714966832237 time 714966571821 state 0x1 ref 1 00000100:00100000:0.0:1789760704.728558:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714966832237 time 715069593104 state 0x0 ref 1 00000100:00100000:0.0:1789760704.728570:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493644672:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760704.728623:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21282816, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa807 00010000:00010000:0.0:1789760704.728631:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0443e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.728646:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760704.728651:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760704.728657:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0443e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.728672:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0443e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760704.782654:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972515 is committed 00002000:00010000:1.0:1789760704.783017:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21282816 -> 21286912 00010000:00010000:1.0:1789760704.783033:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0443e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.783047:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0443e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.783056:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0443e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.783063:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760704.783067:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0443e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.783076:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0443e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.783101:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0443e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760704.783108:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0443e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760704.783210:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493644672:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54636us (54985us total) trans 4294972515 rc 0/0 00000100:00100000:1.0:1789760704.783220:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5226 00000100:00100000:1.0:1789760704.783230:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714966832237 time 715069593104 state 0x2 ref 0 00000100:00100000:1.0:1789760704.793066:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760704.793137:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493644800 00000100:00100000:2.0:1789760704.793147:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493644800 00000100:00100000:2.0:1789760704.793198:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 714966832237 time 715069593104 state 0x0 ref 1 00000100:00100000:2.0:1789760704.793214:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715069853520 time 715069593104 state 0x1 ref 1 00000100:00100000:2.0:1789760704.793243:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715069853520 time 715134279032 state 0x0 ref 1 00000100:00100000:2.0:1789760704.793255:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493644800:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760704.793297:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21286912, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa807 00010000:00010000:2.0:1789760704.793306:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04445 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.793321:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760704.793325:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.793331:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04445 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.793346:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04445 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760704.842379:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972516 is committed 00002000:00010000:2.0:1789760704.842472:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21286912 -> 21291008 00010000:00010000:2.0:1789760704.842489:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04445 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.842502:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04445 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.842516:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04445 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.842524:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760704.842529:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04445 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.842537:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04445 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.842556:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04445 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.842564:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04445 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.842742:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493644800:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49483us (49680us total) trans 4294972516 rc 0/0 00000100:00100000:2.0:1789760704.842763:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5227 00000100:00100000:2.0:1789760704.842772:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715069853520 time 715134279032 state 0x2 ref 0 00000100:00100000:1.0:1789760704.846738:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760704.846823:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493644928 00000100:00100000:0.0:1789760704.846832:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493644928 00000100:00100000:0.0:1789760704.846883:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715069853520 time 715134279032 state 0x0 ref 1 00000100:00100000:0.0:1789760704.846899:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715134539448 time 715134279032 state 0x1 ref 1 00000100:00100000:0.0:1789760704.846932:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715134539448 time 715187967437 state 0x0 ref 1 00000100:00100000:0.0:1789760704.846943:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493644928:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760704.847016:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21291008, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa807 00010000:00010000:0.0:1789760704.847026:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0444c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.847047:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760704.847052:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760704.847058:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0444c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.847073:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0444c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760704.872516:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972517 is committed 00002000:00010000:0.0:1789760704.872596:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21291008 -> 21295104 00010000:00010000:0.0:1789760704.872611:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0444c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.872623:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0444c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.872632:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0444c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.872640:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760704.872644:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0444c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.872653:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0444c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.872671:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0444c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.872679:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0444c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760704.872810:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493644928:12345-192.168.203.60@tcp:4:dd.0 Request processed in 25857us (26070us total) trans 4294972517 rc 0/0 00000100:00100000:0.0:1789760704.872821:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5228 00000100:00100000:0.0:1789760704.872830:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715134539448 time 715187967437 state 0x2 ref 0 00000100:00100000:2.0:1789760704.877114:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760704.877191:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493645056 00000100:00100000:3.0:1789760704.877200:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493645056 00000100:00100000:3.0:1789760704.877319:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715134539448 time 715187967437 state 0x0 ref 1 00000100:00100000:3.0:1789760704.877335:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715188227853 time 715187967437 state 0x1 ref 1 00000100:00100000:3.0:1789760704.877367:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715188227853 time 715218401676 state 0x0 ref 1 00000100:00100000:3.0:1789760704.877380:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493645056:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760704.877430:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21295104, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa807 00010000:00010000:3.0:1789760704.877438:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04453 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.877455:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760704.877459:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760704.877465:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04453 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760704.877480:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04453 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760704.920593:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972518 is committed 00002000:00010000:0.0:1789760704.920940:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21295104 -> 21299200 00010000:00010000:0.0:1789760704.920955:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04453 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.920998:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04453 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.921008:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04453 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.921015:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760704.921020:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04453 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.921029:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04453 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.921053:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04453 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.921062:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04453 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760704.921144:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493645056:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43761us (44034us total) trans 4294972518 rc 0/0 00000100:00100000:0.0:1789760704.921155:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5229 00000100:00100000:0.0:1789760704.921165:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715188227853 time 715218401676 state 0x2 ref 0 00000100:00100000:1.0:1789760704.924942:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760704.927507:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493645184 00000100:00100000:3.0:1789760704.927523:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493645184 00000100:00100000:2.0:1789760704.934190:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715188227853 time 715218401676 state 0x0 ref 1 00000100:00100000:2.0:1789760704.934212:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715218662092 time 715218401676 state 0x1 ref 1 00000100:00100000:2.0:1789760704.934260:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715218662092 time 715275296037 state 0x0 ref 1 00000100:00100000:2.0:1789760704.934278:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493645184:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760704.934331:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21299200, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa807 00010000:00010000:2.0:1789760704.934340:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0445a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.934356:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760704.934361:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.934367:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0445a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.934382:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0445a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760705.019684:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972519 is committed 00002000:00010000:0.0:1789760705.019762:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21299200 -> 21303296 00010000:00010000:0.0:1789760705.019785:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0445a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.019799:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0445a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.019808:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0445a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.019821:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760705.019826:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0445a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.019840:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0445a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.019859:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0445a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.019867:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0445a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760705.019957:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493645184:12345-192.168.203.60@tcp:4:dd.0 Request processed in 85681us (95021us total) trans 4294972519 rc 0/0 00000100:00100000:0.0:1789760705.020002:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5230 00000100:00100000:0.0:1789760705.020011:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715218662092 time 715275296037 state 0x2 ref 0 00000100:00100000:0.0:1789760705.030728:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760705.030843:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493645312 00000100:00100000:2.0:1789760705.030853:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493645312 00000100:00080000:1.0:1789760705.030911:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760705.030936:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715218662092 time 715275296037 state 0x0 ref 1 00000100:00100000:2.0:1789760705.030952:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715275556453 time 715275296037 state 0x1 ref 1 00000100:00100000:2.0:1789760705.031000:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715275556453 time 715372035460 state 0x0 ref 1 00000100:00100000:2.0:1789760705.031016:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493645312:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760705.031067:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21303296, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa807 00010000:00010000:2.0:1789760705.031078:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04461 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.031093:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760705.031097:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.031103:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04461 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.031118:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04461 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760705.077354:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972520 is committed 00002000:00010000:0.0:1789760705.077462:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21303296 -> 21307392 00010000:00010000:0.0:1789760705.077478:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04461 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.077491:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04461 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.077501:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04461 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.077514:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760705.077519:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04461 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.077528:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04461 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.077547:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04461 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.077555:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04461 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760705.077654:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493645312:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46637us (46930us total) trans 4294972520 rc 0/0 00000100:00100000:0.0:1789760705.077664:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5231 00000100:00100000:0.0:1789760705.077674:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715275556453 time 715372035460 state 0x2 ref 0 00000100:00100000:2.0:1789760705.080919:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760705.081002:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493645440 00000100:00100000:3.0:1789760705.081012:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493645440 00000100:00100000:3.0:1789760705.081061:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715275556453 time 715372035460 state 0x0 ref 1 00000100:00100000:3.0:1789760705.081112:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715372295876 time 715372035460 state 0x1 ref 1 00000100:00100000:3.0:1789760705.081138:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715372295876 time 715422173476 state 0x0 ref 1 00000100:00100000:3.0:1789760705.081150:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493645440:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760705.081204:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21307392, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa807 00010000:00010000:3.0:1789760705.081212:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04468 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.081227:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760705.081292:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760705.081299:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04468 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.081318:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04468 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.082522:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760705.110383:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972521 is committed 00002000:00010000:0.0:1789760705.111008:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21307392 -> 21311488 00010000:00010000:0.0:1789760705.111024:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04468 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.111035:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04468 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.111050:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04468 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.111059:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760705.111064:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04468 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.111074:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04468 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.111095:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04468 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.111103:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04468 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760705.111188:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493645440:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30035us (30271us total) trans 4294972521 rc 0/0 00000100:00100000:0.0:1789760705.111199:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5232 00000100:00100000:0.0:1789760705.111208:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715372295876 time 715422173476 state 0x2 ref 0 00000100:00100000:2.0:1789760705.114154:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760705.114337:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493645568 00000100:00100000:0.0:1789760705.114348:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493645568 00000100:00100000:0.0:1789760705.114407:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715372295876 time 715422173476 state 0x0 ref 1 00000100:00100000:0.0:1789760705.114425:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715422433892 time 715422173476 state 0x1 ref 1 00000100:00100000:0.0:1789760705.114455:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715422433892 time 715455490538 state 0x0 ref 1 00000100:00100000:0.0:1789760705.114467:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493645568:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760705.114515:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21311488, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa807 00010000:00010000:0.0:1789760705.114524:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0446f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.114541:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760705.114546:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760705.114554:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0446f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.114569:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0446f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760705.155430:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972522 is committed 00002000:00010000:0.0:1789760705.155613:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21311488 -> 21315584 00010000:00010000:0.0:1789760705.155631:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0446f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.155644:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0446f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.155654:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0446f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.155662:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760705.155668:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0446f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.155680:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0446f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.155704:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0446f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.155711:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0446f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760705.155945:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493645568:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41476us (41796us total) trans 4294972522 rc 0/0 00000100:00100000:0.0:1789760705.155957:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5233 00000100:00100000:0.0:1789760705.156008:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715422433892 time 715455490538 state 0x2 ref 0 00000100:00100000:1.0:1789760705.166316:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760705.166995:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493645696 00000100:00100000:0.0:1789760705.167006:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493645696 00000100:00100000:0.0:1789760705.167061:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715422433892 time 715455490538 state 0x0 ref 1 00000100:00100000:0.0:1789760705.167076:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715455750954 time 715455490538 state 0x1 ref 1 00000100:00100000:0.0:1789760705.167224:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715455750954 time 715508257573 state 0x0 ref 1 00000100:00100000:0.0:1789760705.167238:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493645696:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760705.167289:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21315584, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa807 00010000:00010000:0.0:1789760705.167299:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04476 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.167315:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760705.167319:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760705.167326:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04476 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.167340:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04476 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760705.202370:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972523 is committed 00002000:00010000:0.0:1789760705.202491:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21315584 -> 21319680 00010000:00010000:0.0:1789760705.202506:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04476 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.202520:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04476 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.202529:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04476 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.202535:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760705.202540:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04476 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.202549:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04476 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.202584:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04476 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.202591:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04476 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760705.202685:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493645696:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35444us (36372us total) trans 4294972523 rc 0/0 00000100:00100000:0.0:1789760705.202696:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5234 00000100:00100000:0.0:1789760705.202706:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715455750954 time 715508257573 state 0x2 ref 0 00000100:00100000:1.0:1789760705.207386:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760705.207538:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493645824 00000100:00100000:2.0:1789760705.207550:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493645824 00000100:00100000:2.0:1789760705.207607:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715455750954 time 715508257573 state 0x0 ref 1 00000100:00100000:2.0:1789760705.207624:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715508517989 time 715508257573 state 0x1 ref 1 00000100:00100000:2.0:1789760705.207688:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715508517989 time 715548723434 state 0x0 ref 1 00000100:00100000:2.0:1789760705.207702:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493645824:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760705.207765:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21319680, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa807 00010000:00010000:2.0:1789760705.207774:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0447d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.207789:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760705.207794:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.207801:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0447d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.207824:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0447d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760705.269657:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972524 is committed 00002000:00010000:0.0:1789760705.269771:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21319680 -> 21323776 00010000:00010000:0.0:1789760705.269786:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0447d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.269809:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0447d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.269820:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0447d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.269827:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760705.269833:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0447d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.269846:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0447d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.269865:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0447d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.269874:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0447d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760705.270010:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493645824:12345-192.168.203.60@tcp:4:dd.0 Request processed in 62304us (62627us total) trans 4294972524 rc 0/0 00000100:00100000:0.0:1789760705.270021:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5235 00000100:00100000:0.0:1789760705.270031:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715508517989 time 715548723434 state 0x2 ref 0 00000100:00100000:3.0:1789760705.274454:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760705.274536:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493645952 00000100:00100000:0.0:1789760705.274544:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493645952 00000100:00100000:0.0:1789760705.274597:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715508517989 time 715548723434 state 0x0 ref 1 00000100:00100000:0.0:1789760705.274611:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715548983850 time 715548723434 state 0x1 ref 1 00000100:00100000:0.0:1789760705.274643:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715548983850 time 715615679474 state 0x0 ref 1 00000100:00100000:0.0:1789760705.274655:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493645952:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760705.274701:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21323776, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa807 00010000:00010000:0.0:1789760705.274709:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04484 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.274724:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760705.274728:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760705.274734:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04484 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.274760:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04484 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760705.376282:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972525 is committed 00002000:00010000:3.0:1789760705.376423:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21323776 -> 21327872 00010000:00010000:3.0:1789760705.376437:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04484 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.376450:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04484 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.376458:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04484 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.376465:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760705.376470:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04484 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.376479:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04484 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.376503:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04484 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.376524:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04484 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760705.376613:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493645952:12345-192.168.203.60@tcp:4:dd.0 Request processed in 101956us (102165us total) trans 4294972525 rc 0/0 00000100:00100000:3.0:1789760705.376623:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5236 00000100:00100000:3.0:1789760705.376632:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715548983850 time 715615679474 state 0x2 ref 0 00000100:00100000:3.0:1789760705.381256:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760705.381404:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493646080 00000100:00100000:0.0:1789760705.381415:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493646080 00000100:00100000:0.0:1789760705.381469:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715548983850 time 715615679474 state 0x0 ref 1 00000100:00100000:0.0:1789760705.381485:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715615939890 time 715615679474 state 0x1 ref 1 00000100:00100000:0.0:1789760705.381533:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715615939890 time 715722568209 state 0x0 ref 1 00000100:00100000:0.0:1789760705.381552:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493646080:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760705.381598:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21327872, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa807 00010000:00010000:0.0:1789760705.381607:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e0448b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.381623:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760705.381627:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760705.381633:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e0448b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.381653:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e0448b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760705.436641:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972526 is committed 00002000:00010000:0.0:1789760705.437006:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21327872 -> 21331968 00010000:00010000:0.0:1789760705.437022:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e0448b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.437035:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e0448b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.437043:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e0448b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.437050:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760705.437055:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e0448b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.437063:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e0448b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.437081:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e0448b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.437088:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e0448b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760705.437177:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493646080:12345-192.168.203.60@tcp:4:dd.0 Request processed in 55622us (55934us total) trans 4294972526 rc 0/0 00000100:00100000:0.0:1789760705.437188:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5237 00000100:00100000:0.0:1789760705.437198:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715615939890 time 715722568209 state 0x2 ref 0 00000100:00100000:1.0:1789760705.440865:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760705.441154:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493646208 00000100:00100000:2.0:1789760705.441165:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493646208 00000100:00100000:2.0:1789760705.441218:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715615939890 time 715722568209 state 0x0 ref 1 00000100:00100000:2.0:1789760705.441245:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715722828625 time 715722568209 state 0x1 ref 1 00000100:00100000:2.0:1789760705.441284:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715722828625 time 715782319003 state 0x0 ref 1 00000100:00100000:2.0:1789760705.441296:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493646208:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760705.441345:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21331968, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa807 00010000:00010000:2.0:1789760705.441354:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04492 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.441370:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760705.441375:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.441381:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04492 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.441395:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04492 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760705.521092:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972527 is committed 00002000:00010000:2.0:1789760705.521307:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21331968 -> 21336064 00010000:00010000:2.0:1789760705.521323:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04492 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.521335:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04492 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.521344:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04492 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.521352:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760705.521357:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04492 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.521366:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04492 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.521384:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04492 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.521392:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04492 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.521649:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493646208:12345-192.168.203.60@tcp:4:dd.0 Request processed in 80349us (80845us total) trans 4294972527 rc 0/0 00000100:00100000:2.0:1789760705.521663:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5238 00000100:00100000:2.0:1789760705.521673:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715722828625 time 715782319003 state 0x2 ref 0 00000100:00100000:3.0:1789760705.529424:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760705.529608:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493646336 00000100:00100000:1.0:1789760705.529619:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493646336 00000100:00100000:1.0:1789760705.529672:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715722828625 time 715782319003 state 0x0 ref 1 00000100:00100000:1.0:1789760705.529687:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715782579419 time 715782319003 state 0x1 ref 1 00000100:00100000:1.0:1789760705.529708:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715782579419 time 715870743934 state 0x0 ref 1 00000100:00100000:1.0:1789760705.529721:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493646336:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760705.529779:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21336064, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa807 00010000:00010000:1.0:1789760705.529788:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04499 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.529804:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760705.529809:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760705.529815:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04499 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760705.529830:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04499 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760705.676719:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972528 is committed 00002000:00010000:3.0:1789760705.676991:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21336064 -> 21340160 00010000:00010000:3.0:1789760705.677006:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04499 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.677020:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04499 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.677030:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04499 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.677038:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760705.677044:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04499 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.677054:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04499 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.677076:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04499 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.677084:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04499 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760705.677178:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493646336:12345-192.168.203.60@tcp:4:dd.0 Request processed in 147454us (147757us total) trans 4294972528 rc 0/0 00000100:00100000:3.0:1789760705.677189:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5239 00000100:00100000:3.0:1789760705.677200:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715782579419 time 715870743934 state 0x2 ref 0 00000100:00100000:3.0:1789760705.682099:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760705.682347:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493646464 00000100:00100000:0.0:1789760705.682356:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493646464 00000100:00100000:0.0:1789760705.682410:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715782579419 time 715870743934 state 0x0 ref 1 00000100:00100000:0.0:1789760705.682425:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715871004350 time 715870743934 state 0x1 ref 1 00000100:00100000:0.0:1789760705.682456:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715871004350 time 716023491265 state 0x0 ref 1 00000100:00100000:0.0:1789760705.682478:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493646464:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760705.682528:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21340160, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0xa807 00010000:00010000:0.0:1789760705.682537:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e044a0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.682553:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760705.682557:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760705.682563:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e044a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.682577:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e044a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760705.842942:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972529 is committed 00002000:00010000:3.0:1789760705.843282:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21340160 -> 21344256 00002000:00010000:3.0:1789760705.843288:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760704 -> 1789760705 00002000:00010000:3.0:1789760705.843291:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760704 -> 1789760705 00010000:00010000:3.0:1789760705.843305:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e044a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.843318:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e044a0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.843333:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e044a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.843341:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760705.843346:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e044a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.843356:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e044a0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.843379:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e044a0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.843387:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e044a0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760705.843483:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493646464:12345-192.168.203.60@tcp:4:dd.0 Request processed in 161004us (161389us total) trans 4294972529 rc 0/0 00000100:00100000:3.0:1789760705.843494:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5240 00000100:00100000:3.0:1789760705.843505:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715871004350 time 716023491265 state 0x2 ref 0 00000100:00100000:2.0:1789760705.852575:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760705.852664:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493646592 00000100:00100000:0.0:1789760705.852673:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493646592 00000100:00100000:0.0:1789760705.852722:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 715871004350 time 716023491265 state 0x0 ref 1 00000100:00100000:0.0:1789760705.852736:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716023751681 time 716023491265 state 0x1 ref 1 00000100:00100000:0.0:1789760705.852769:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716023751681 time 716193805290 state 0x0 ref 1 00000100:00100000:0.0:1789760705.852780:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493646592:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760705.852834:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21344256, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa807 00010000:00010000:0.0:1789760705.852842:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e044a7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.852857:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760705.852861:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760705.852867:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e044a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.852880:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e044a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760705.982515:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972530 is committed 00002000:00010000:3.0:1789760705.982629:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21344256 -> 21348352 00010000:00010000:3.0:1789760705.982645:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e044a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.982665:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e044a7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.982674:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e044a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.982683:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760705.982687:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e044a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.982697:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e044a7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.982717:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e044a7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760705.982725:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e044a7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760705.982829:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493646592:12345-192.168.203.60@tcp:4:dd.0 Request processed in 130045us (130258us total) trans 4294972530 rc 0/0 00000100:00100000:3.0:1789760705.982846:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5241 00000100:00100000:3.0:1789760705.982857:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716023751681 time 716193805290 state 0x2 ref 0 00000100:00100000:0.0:1789760705.992931:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760705.993081:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493646720 00000100:00100000:2.0:1789760705.993092:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493646720 00000100:00100000:2.0:1789760705.993147:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716023751681 time 716193805290 state 0x0 ref 1 00000100:00100000:2.0:1789760705.993162:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716194065706 time 716193805290 state 0x1 ref 1 00000100:00100000:2.0:1789760705.993187:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716194065706 time 716334222878 state 0x0 ref 1 00000100:00100000:2.0:1789760705.993200:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493646720:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760705.993246:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21348352, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa807 00010000:00010000:2.0:1789760705.993265:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e044ae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.993280:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760705.993285:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.993291:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e044ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.993305:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e044ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760706.080493:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972531 is committed 00002000:00010000:2.0:1789760706.082072:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21348352 -> 21352448 00010000:00010000:2.0:1789760706.082089:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e044ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.082102:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e044ae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.082111:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e044ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.082118:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760706.082123:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e044ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.082131:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e044ae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.082149:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e044ae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.082156:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e044ae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760706.082262:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493646720:12345-192.168.203.60@tcp:4:dd.0 Request processed in 89059us (89428us total) trans 4294972531 rc 0/0 00000100:00100000:2.0:1789760706.082272:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5242 00000100:00100000:2.0:1789760706.082282:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716194065706 time 716334222878 state 0x2 ref 0 00000100:00100000:3.0:1789760706.085347:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760706.085491:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493646848 00000100:00100000:0.0:1789760706.085501:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493646848 00000100:00100000:0.0:1789760706.085555:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716194065706 time 716334222878 state 0x0 ref 1 00000100:00100000:0.0:1789760706.085569:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716334483294 time 716334222878 state 0x1 ref 1 00000100:00100000:0.0:1789760706.085596:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716334483294 time 716426632845 state 0x0 ref 1 00000100:00100000:0.0:1789760706.085607:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493646848:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760706.085652:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21352448, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa807 00010000:00010000:0.0:1789760706.085660:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e044b5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.085675:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760706.085679:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760706.085685:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e044b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.085698:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e044b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.108141:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760706.169165:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21352448 -> 21356544 00010000:00010000:1.0:1789760706.169187:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e044b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.169201:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e044b5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.169211:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e044b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.169223:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760706.169228:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e044b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.169236:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e044b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.169255:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e044b5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.169262:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e044b5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760706.169363:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493646848:12345-192.168.203.60@tcp:4:dd.0 Request processed in 83750us (84019us total) trans 4294972532 rc 0/0 00000100:00100000:1.0:1789760706.169373:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5243 00000100:00100000:1.0:1789760706.169383:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716334483294 time 716426632845 state 0x2 ref 0 00000001:00080000:0.0:1789760706.169768:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972532 is committed 00000100:00100000:3.0:1789760706.173441:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760706.173540:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493646976 00000100:00100000:0.0:1789760706.173549:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493646976 00000100:00100000:0.0:1789760706.173604:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716334483294 time 716426632845 state 0x0 ref 1 00000100:00100000:0.0:1789760706.173619:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716426893261 time 716426632845 state 0x1 ref 1 00000100:00100000:0.0:1789760706.173650:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716426893261 time 716514686109 state 0x0 ref 1 00000100:00100000:0.0:1789760706.173664:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493646976:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760706.173713:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21356544, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa807 00010000:00010000:0.0:1789760706.173721:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e044bc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.173735:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760706.173739:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760706.173755:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e044bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.173769:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e044bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760706.352157:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21356544 -> 21360640 00010000:00010000:2.0:1789760706.352183:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e044bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.352196:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e044bc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.352206:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e044bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.352214:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760706.352219:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e044bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.352230:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e044bc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.352249:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e044bc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.352257:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e044bc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760706.352357:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493646976:12345-192.168.203.60@tcp:4:dd.0 Request processed in 178691us (178920us total) trans 4294972533 rc 0/0 00000100:00100000:2.0:1789760706.352368:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5244 00000100:00100000:2.0:1789760706.352378:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716426893261 time 716514686109 state 0x2 ref 0 00000001:00080000:0.0:1789760706.352453:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972533 is committed 00000100:00100000:2.0:1789760706.359049:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760706.359211:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493647104 00000100:00100000:3.0:1789760706.359221:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493647104 00000100:00100000:3.0:1789760706.359276:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716426893261 time 716514686109 state 0x0 ref 1 00000100:00100000:3.0:1789760706.359291:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716514946525 time 716514686109 state 0x1 ref 1 00000100:00100000:3.0:1789760706.359322:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716514946525 time 716700357513 state 0x0 ref 1 00000100:00100000:3.0:1789760706.359335:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493647104:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760706.359381:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21360640, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa807 00010000:00010000:3.0:1789760706.359391:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e044c3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.359406:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760706.359410:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760706.359416:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e044c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.359430:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e044c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760706.448827:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21360640 -> 21364736 00010000:00010000:2.0:1789760706.448849:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e044c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.448862:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e044c3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.448872:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e044c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.448880:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760706.448885:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e044c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.448894:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e044c3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.448927:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e044c3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.448936:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e044c3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760706.449068:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493647104:12345-192.168.203.60@tcp:4:dd.0 Request processed in 89729us (90022us total) trans 4294972534 rc 0/0 00000100:00100000:2.0:1789760706.449080:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5245 00000100:00100000:2.0:1789760706.449089:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716514946525 time 716700357513 state 0x2 ref 0 00000001:00080000:0.0:1789760706.449423:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972534 is committed 00000100:00100000:1.0:1789760706.456308:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760706.456799:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493647232 00000100:00100000:2.0:1789760706.456809:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493647232 00000100:00100000:2.0:1789760706.456866:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716514946525 time 716700357513 state 0x0 ref 1 00000100:00100000:2.0:1789760706.456881:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716700617929 time 716700357513 state 0x1 ref 1 00000100:00100000:2.0:1789760706.456927:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716700617929 time 716797961844 state 0x0 ref 1 00000100:00100000:2.0:1789760706.456941:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493647232:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760706.457021:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21364736, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa807 00010000:00010000:2.0:1789760706.457030:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e044ca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.457046:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760706.457050:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760706.457056:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e044ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.457071:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e044ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760706.538493:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972535 is committed 00002000:00010000:0.0:1789760706.541335:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21364736 -> 21368832 00010000:00010000:0.0:1789760706.541357:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e044ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.541372:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e044ca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.541381:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e044ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.541393:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760706.541398:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e044ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.541408:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e044ca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.541432:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e044ca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.541441:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e044ca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760706.541553:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493647232:12345-192.168.203.60@tcp:4:dd.0 Request processed in 84610us (85249us total) trans 4294972535 rc 0/0 00000100:00100000:0.0:1789760706.541564:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5246 00000100:00100000:0.0:1789760706.541573:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716700617929 time 716797961844 state 0x2 ref 0 00000100:00100000:2.0:1789760706.551585:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760706.551660:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493647360 00000100:00100000:1.0:1789760706.551668:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493647360 00000100:00100000:1.0:1789760706.551715:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716700617929 time 716797961844 state 0x0 ref 1 00000100:00100000:1.0:1789760706.551730:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716798222260 time 716797961844 state 0x1 ref 1 00000100:00100000:1.0:1789760706.551762:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716798222260 time 716892798404 state 0x0 ref 1 00000100:00100000:1.0:1789760706.551773:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493647360:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760706.551825:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21368832, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa807 00010000:00010000:1.0:1789760706.551834:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e044d1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.551850:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760706.551855:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760706.551860:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e044d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.551874:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e044d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760706.663528:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760706.663572:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760706.663610:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760706.663641:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493647744 00000100:00100000:1.0:1789760706.663650:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493647744 00000100:00100000:1.0:1789760706.663705:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493647616 00000100:00100000:1.0:1789760706.663708:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493647616 00000100:00100000:1.0:1789760706.663735:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 447 00000100:00100000:1.0:1789760706.663755:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d89880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2370:x1876699493647744:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:3.0:1789760706.663868:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493647488 00000100:00100000:3.0:1789760706.663873:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493647488 00000100:00100000:3.0:1789760706.663897:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 385 00000100:00100000:3.0:1789760706.663903:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d4e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2370:x1876699493647488:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:0.0:1789760706.664096:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 448 00000100:00100000:0.0:1789760706.664106:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8556300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2370:x1876699493647616:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:1.0:1789760706.669787:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d89880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2370:x1876699493647744:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 6020us (6200us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760706.669811:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 447 00002000:00010000:1.0:1789760706.669891:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21368832 -> 21372928 00010000:00010000:1.0:1789760706.669905:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e044d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.669928:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e044d1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.669937:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e044d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.669944:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760706.669948:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e044d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.669957:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e044d1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.669993:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e044d1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.670000:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e044d1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760706.671201:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d4e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2370:x1876699493647488:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 7288us (7673us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760706.671214:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 385 00000100:00080000:3.0:1789760706.673459:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760706.674057:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8556300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2370:x1876699493647616:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 9944us (10437us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760706.674070:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 448 00000001:00080000:0.0:1789760706.674119:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972536 is committed 00000100:00100000:1.0:1789760706.674411:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493647360:12345-192.168.203.60@tcp:4:dd.0 Request processed in 122636us (122832us total) trans 4294972536 rc 0/0 00000100:00100000:1.0:1789760706.674423:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5247 00000100:00100000:1.0:1789760706.674433:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716798222260 time 716892798404 state 0x2 ref 0 00000100:00100000:2.0:1789760706.682466:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760706.682567:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493647872 00000100:00100000:0.0:1789760706.682576:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493647872 00000100:00100000:0.0:1789760706.682631:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716798222260 time 716892798404 state 0x0 ref 1 00000100:00100000:0.0:1789760706.683425:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716893058820 time 716892798404 state 0x1 ref 1 00000100:00100000:2.0:1789760706.688388:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716893058820 time 717029410725 state 0x0 ref 1 00000100:00100000:2.0:1789760706.688423:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493647872:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760706.688479:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21372928, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0xa807 00010000:00010000:2.0:1789760706.688488:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e044d8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.688504:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760706.688509:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760706.688516:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e044d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.688530:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e044d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760706.829163:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972537 is committed 00002000:00010000:2.0:1789760706.829467:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21372928 -> 21377024 00002000:00010000:2.0:1789760706.829472:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760705 -> 1789760706 00002000:00010000:2.0:1789760706.829475:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760705 -> 1789760706 00010000:00010000:2.0:1789760706.829488:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e044d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.829502:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e044d8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.829511:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e044d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.829535:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760706.829540:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e044d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.829550:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e044d8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.829570:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e044d8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.829578:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e044d8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760706.829673:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493647872:12345-192.168.203.60@tcp:4:dd.0 Request processed in 141261us (147212us total) trans 4294972537 rc 0/0 00000100:00100000:2.0:1789760706.829684:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5248 00000100:00100000:2.0:1789760706.829695:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716893058820 time 717029410725 state 0x2 ref 0 00000100:00100000:2.0:1789760706.836153:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760706.836277:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493648000 00000100:00100000:3.0:1789760706.836287:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493648000 00000100:00100000:3.0:1789760706.836340:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 716893058820 time 717029410725 state 0x0 ref 1 00000100:00100000:3.0:1789760706.836354:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717029671141 time 717029410725 state 0x1 ref 1 00000100:00100000:3.0:1789760706.836386:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717029671141 time 717177422407 state 0x0 ref 1 00000100:00100000:3.0:1789760706.836399:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493648000:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760706.836445:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21377024, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa807 00010000:00010000:3.0:1789760706.836454:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e044df lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.836470:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760706.836475:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760706.836481:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e044df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.836495:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e044df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760706.977431:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972538 is committed 00002000:00010000:1.0:1789760706.977663:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21377024 -> 21381120 00010000:00010000:1.0:1789760706.977678:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e044df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.977692:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e044df lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.977699:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e044df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.977706:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760706.977711:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e044df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.977720:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e044df lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.977749:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e044df lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760706.977757:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e044df lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760706.978111:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493648000:12345-192.168.203.60@tcp:4:dd.0 Request processed in 141708us (141962us total) trans 4294972538 rc 0/0 00000100:00100000:1.0:1789760706.978121:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5249 00000100:00100000:1.0:1789760706.978131:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717029671141 time 717177422407 state 0x2 ref 0 00000100:00100000:0.0:1789760706.985485:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760706.988351:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493648128 00000100:00100000:3.0:1789760706.988366:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493648128 00000100:00100000:3.0:1789760706.988428:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717029671141 time 717177422407 state 0x0 ref 1 00000100:00100000:3.0:1789760706.988443:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717177682823 time 717177422407 state 0x1 ref 1 00000100:00100000:3.0:1789760706.988473:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717177682823 time 717329509031 state 0x0 ref 1 00000100:00100000:3.0:1789760706.988487:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493648128:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760706.988637:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21381120, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa807 00010000:00010000:3.0:1789760706.988646:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e044e6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.988660:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760706.988665:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760706.988671:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e044e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760706.988685:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e044e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760707.052697:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972539 is committed 00002000:00010000:3.0:1789760707.052820:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21381120 -> 21385216 00010000:00010000:3.0:1789760707.052951:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e044e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.052995:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e044e6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.053004:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e044e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.053011:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760707.053017:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e044e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.053026:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e044e6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.053045:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e044e6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.053053:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e044e6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760707.053150:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493648128:12345-192.168.203.60@tcp:4:dd.0 Request processed in 64659us (67670us total) trans 4294972539 rc 0/0 00000100:00100000:3.0:1789760707.053162:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5250 00000100:00100000:3.0:1789760707.053172:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717177682823 time 717329509031 state 0x2 ref 0 00000100:00100000:2.0:1789760707.058068:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760707.058897:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493648256 00000100:00100000:1.0:1789760707.058907:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493648256 00000100:00100000:1.0:1789760707.059030:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717177682823 time 717329509031 state 0x0 ref 1 00000100:00100000:1.0:1789760707.059046:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717329769447 time 717329509031 state 0x1 ref 1 00000100:00100000:1.0:1789760707.059079:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717329769447 time 717400115096 state 0x0 ref 1 00000100:00100000:1.0:1789760707.059092:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493648256:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760707.059136:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21385216, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa807 00010000:00010000:1.0:1789760707.059144:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e044ed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.059159:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760707.059163:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760707.059181:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e044ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.059195:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e044ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760707.069605:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1789760707.134265:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760707.193575:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972540 is committed 00002000:00010000:1.0:1789760707.193841:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21385216 -> 21389312 00010000:00010000:1.0:1789760707.193856:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e044ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.193874:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e044ed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.193883:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e044ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.193890:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760707.193894:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e044ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.193903:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e044ed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.193923:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e044ed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.193930:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e044ed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760707.194064:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493648256:12345-192.168.203.60@tcp:4:dd.0 Request processed in 134968us (135998us total) trans 4294972540 rc 0/0 00000100:00100000:1.0:1789760707.194076:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5251 00000100:00100000:1.0:1789760707.194087:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717329769447 time 717400115096 state 0x2 ref 0 00000100:00100000:2.0:1789760707.197327:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760707.197425:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493648384 00000100:00100000:0.0:1789760707.197434:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493648384 00000100:00100000:0.0:1789760707.197482:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717329769447 time 717400115096 state 0x0 ref 1 00000100:00100000:0.0:1789760707.197496:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717400375512 time 717400115096 state 0x1 ref 1 00000100:00100000:0.0:1789760707.197522:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717400375512 time 717538558147 state 0x0 ref 1 00000100:00100000:0.0:1789760707.197532:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493648384:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760707.197577:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21389312, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa807 00010000:00010000:0.0:1789760707.197585:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e044f4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.197600:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760707.197605:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760707.197610:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e044f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.197624:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e044f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760707.311671:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21389312 -> 21393408 00010000:00010000:3.0:1789760707.311692:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e044f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.311706:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e044f4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.311720:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e044f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.311728:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760707.311743:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e044f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.311754:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e044f4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.311773:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e044f4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.311781:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e044f4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760707.311881:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493648384:12345-192.168.203.60@tcp:4:dd.0 Request processed in 114344us (114559us total) trans 4294972541 rc 0/0 00000100:00100000:3.0:1789760707.311892:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5252 00000100:00100000:3.0:1789760707.311901:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717400375512 time 717538558147 state 0x2 ref 0 00000001:00080000:0.0:1789760707.312526:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972541 is committed 00000100:00100000:1.0:1789760707.320163:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760707.320368:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493648512 00000100:00100000:2.0:1789760707.320378:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493648512 00000100:00100000:2.0:1789760707.320436:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717400375512 time 717538558147 state 0x0 ref 1 00000100:00100000:2.0:1789760707.320452:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717538818563 time 717538558147 state 0x1 ref 1 00000100:00100000:2.0:1789760707.320483:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717538818563 time 717661519169 state 0x0 ref 1 00000100:00100000:2.0:1789760707.320496:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8555500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493648512:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760707.320545:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21393408, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa807 00010000:00010000:2.0:1789760707.320554:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e044fb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.320569:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760707.320574:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760707.320580:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e044fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.320595:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e044fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760707.392683:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972542 is committed 00002000:00010000:1.0:1789760707.392862:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21393408 -> 21397504 00010000:00010000:1.0:1789760707.392878:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e044fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.392891:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e044fb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.392900:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e044fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.392908:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760707.392912:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e044fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.392921:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e044fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.392941:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e044fb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.392949:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e044fb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760707.393095:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8555500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493648512:12345-192.168.203.60@tcp:4:dd.0 Request processed in 72597us (72937us total) trans 4294972542 rc 0/0 00000100:00100000:1.0:1789760707.393107:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5253 00000100:00100000:1.0:1789760707.393116:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717538818563 time 717661519169 state 0x2 ref 0 00000100:00100000:2.0:1789760707.398392:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760707.398471:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493648640 00000100:00100000:3.0:1789760707.398482:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493648640 00000100:00100000:3.0:1789760707.398536:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717538818563 time 717661519169 state 0x0 ref 1 00000100:00100000:3.0:1789760707.398556:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717661779585 time 717661519169 state 0x1 ref 1 00000100:00100000:3.0:1789760707.398589:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717661779585 time 717739624536 state 0x0 ref 1 00000100:00100000:3.0:1789760707.398603:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493648640:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760707.398654:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21397504, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa807 00010000:00010000:3.0:1789760707.398662:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04502 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.398678:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760707.398683:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760707.398700:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04502 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.398715:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04502 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760707.450754:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760707.450779:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620c8554e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3304:1876699519246208:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760707.450860:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760707.451087:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519246208 00000100:00100000:1.0:1789760707.451096:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519246208 00000100:00100000:1.0:1789760707.451165:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 186 00000100:00100000:1.0:1789760707.451174:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519246208:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760707.451196:0:6690:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528154624 pending 1703936 free 3836739584 avail 3828350976 00000100:00100000:1.0:1789760707.451253:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519246208:12345-0@lo:13:osp-pre-0-0.0 Request processed in 79us (392us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760707.451260:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 186 00000100:00080000:0.0:1789760707.452921:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760707.453003:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620c8554e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3304:1876699519246208:0@lo:13:osp-pre-0-0.0 00000001:00080000:3.0:1789760707.466564:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972543 is committed 00002000:00010000:0.0:1789760707.467086:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21397504 -> 21401600 00010000:00010000:0.0:1789760707.467102:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04502 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.467115:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04502 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.467125:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04502 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.467133:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760707.467137:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04502 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.467146:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04502 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.467165:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04502 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.467179:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04502 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760707.467280:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493648640:12345-192.168.203.60@tcp:4:dd.0 Request processed in 68675us (68893us total) trans 4294972543 rc 0/0 00000100:00100000:0.0:1789760707.475952:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5254 00000100:00100000:0.0:1789760707.475983:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717661779585 time 717739624536 state 0x2 ref 0 00000100:00100000:2.0:1789760707.477652:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760707.477725:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493648768 00000100:00100000:1.0:1789760707.477747:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493648768 00000100:00100000:1.0:1789760707.477808:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717661779585 time 717739624536 state 0x0 ref 1 00000100:00100000:1.0:1789760707.477823:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717739884952 time 717739624536 state 0x1 ref 1 00000100:00100000:1.0:1789760707.477851:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717739884952 time 717818886582 state 0x0 ref 1 00000100:00100000:1.0:1789760707.477863:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493648768:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760707.477910:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21401600, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa807 00010000:00010000:1.0:1789760707.477919:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04509 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.477934:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760707.477938:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760707.477944:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04509 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.477993:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04509 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760707.535044:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972544 is committed 00002000:00010000:0.0:1789760707.535376:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21401600 -> 21405696 00010000:00010000:0.0:1789760707.535392:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04509 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.535406:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04509 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.535414:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04509 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.535422:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760707.535426:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04509 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.535434:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04509 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.535460:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04509 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.535469:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04509 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760707.535578:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493648768:12345-192.168.203.60@tcp:4:dd.0 Request processed in 57710us (57926us total) trans 4294972544 rc 0/0 00000100:00100000:0.0:1789760707.535589:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5255 00000100:00100000:0.0:1789760707.535600:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717739884952 time 717818886582 state 0x2 ref 0 00000100:00100000:1.0:1789760707.545634:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760707.545739:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493648896 00000100:00100000:0.0:1789760707.545749:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493648896 00000100:00100000:0.0:1789760707.545809:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717739884952 time 717818886582 state 0x0 ref 1 00000100:00100000:0.0:1789760707.545824:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717819146998 time 717818886582 state 0x1 ref 1 00000100:00100000:0.0:1789760707.545853:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717819146998 time 717886888915 state 0x0 ref 1 00000100:00100000:0.0:1789760707.545865:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8557b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493648896:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760707.545913:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21405696, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa807 00010000:00010000:0.0:1789760707.545922:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04510 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.545936:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760707.545941:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760707.545948:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04510 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.545997:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04510 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760707.622780:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972545 is committed 00002000:00010000:0.0:1789760707.623674:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21405696 -> 21409792 00010000:00010000:0.0:1789760707.623692:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04510 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.623706:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04510 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.623715:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04510 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.623723:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760707.623729:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04510 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.623758:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04510 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.623779:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04510 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.623795:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04510 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760707.623894:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8557b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493648896:12345-192.168.203.60@tcp:4:dd.0 Request processed in 78026us (78265us total) trans 4294972545 rc 0/0 00000100:00100000:0.0:1789760707.623910:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5256 00000100:00100000:0.0:1789760707.623920:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717819146998 time 717886888915 state 0x2 ref 0 00000100:00100000:2.0:1789760707.631905:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760707.632131:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493649024 00000100:00100000:1.0:1789760707.632141:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493649024 00000100:00100000:1.0:1789760707.632197:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717819146998 time 717886888915 state 0x0 ref 1 00000100:00100000:1.0:1789760707.632223:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717887149331 time 717886888915 state 0x1 ref 1 00000100:00100000:1.0:1789760707.632264:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717887149331 time 717973292591 state 0x0 ref 1 00000100:00100000:1.0:1789760707.632279:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493649024:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760707.632337:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21409792, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0xa807 00010000:00010000:1.0:1789760707.632346:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04517 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.632371:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760707.632375:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760707.632382:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04517 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.632397:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04517 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760707.759618:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21409792 -> 21413888 00002000:00010000:3.0:1789760707.759628:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760706 -> 1789760707 00002000:00010000:3.0:1789760707.759631:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760706 -> 1789760707 00010000:00010000:3.0:1789760707.759643:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04517 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.759661:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04517 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.759669:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04517 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.759676:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760707.759680:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04517 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.759690:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04517 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.759711:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04517 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760707.759718:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04517 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760707.759957:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493649024:12345-192.168.203.60@tcp:4:dd.0 Request processed in 127675us (128058us total) trans 4294972546 rc 0/0 00000100:00100000:3.0:1789760707.760003:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5257 00000100:00100000:3.0:1789760707.760012:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717887149331 time 717973292591 state 0x2 ref 0 00000001:00080000:3.0:1789760707.760053:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972546 is committed 00000100:00100000:2.0:1789760707.769613:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760707.769704:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493649152 00000100:00100000:3.0:1789760707.769714:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493649152 00000100:00100000:2.0:1789760707.781673:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717887149331 time 717973292591 state 0x0 ref 1 00000100:00100000:2.0:1789760707.781692:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717973553007 time 717973292591 state 0x1 ref 1 00000100:00100000:2.0:1789760707.781749:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717973553007 time 718122783499 state 0x0 ref 1 00000100:00100000:2.0:1789760707.781771:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493649152:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760707.781834:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21413888, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa807 00010000:00010000:2.0:1789760707.781843:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0451e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.781859:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760707.781864:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760707.781871:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0451e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.781885:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0451e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760707.920653:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972547 is committed 00002000:00010000:1.0:1789760707.921017:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21413888 -> 21417984 00010000:00010000:1.0:1789760707.921032:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0451e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.921045:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0451e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.921054:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0451e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.921061:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760707.921066:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0451e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.921075:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0451e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.921094:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0451e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760707.921103:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0451e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760707.921201:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493649152:12345-192.168.203.60@tcp:4:dd.0 Request processed in 139428us (151585us total) trans 4294972547 rc 0/0 00000100:00100000:1.0:1789760707.921216:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5258 00000100:00100000:1.0:1789760707.921227:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717973553007 time 718122783499 state 0x2 ref 0 00000100:00100000:1.0:1789760707.933445:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760707.933606:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493649280 00000100:00100000:0.0:1789760707.933616:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493649280 00000100:00100000:0.0:1789760707.933670:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 717973553007 time 718122783499 state 0x0 ref 1 00000100:00100000:0.0:1789760707.933685:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718123043915 time 718122783499 state 0x1 ref 1 00000100:00100000:0.0:1789760707.933714:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718123043915 time 718274749917 state 0x0 ref 1 00000100:00100000:0.0:1789760707.933727:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8555180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493649280:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760707.933795:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21417984, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa807 00010000:00010000:0.0:1789760707.933804:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04525 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.933819:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760707.933824:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760707.933830:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04525 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.933844:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04525 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760707.964380:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc5836a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3304:1876699519246336:0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.964456:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760707.964518:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620fb460a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3304:1876699519246848:0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.964545:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760707.964571:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760707.964577:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc5837100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3304:1876699519246720:0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.964599:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760707.964625:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc5835500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3305:1876699519246464:0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.964645:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760707.964659:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620fb461c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519246976:0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.964680:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760707.964697:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789760707.964703:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc5835180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519246592:0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.964724:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760707.964767:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519246336 00000100:00100000:3.0:1789760707.964774:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519246336 00000100:00100000:3.0:1789760707.964821:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 386 00000100:00100000:3.0:1789760707.964830:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d4dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3304:x1876699519246336:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.967225:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d4dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3304:x1876699519246336:12345-0@lo:400:kworker.0 Request processed in 2385us (2757us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760707.967242:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 386 00000100:00100000:3.0:1789760707.967328:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519246848 00000100:00100000:3.0:1789760707.967337:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519246848 00000100:00100000:3.0:1789760707.967387:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519246720 00000100:00100000:3.0:1789760707.967390:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519246720 00000100:00100000:3.0:1789760707.967408:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519246464 00000100:00100000:3.0:1789760707.967410:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519246464 00000100:00100000:3.0:1789760707.967423:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 449 00000100:00100000:3.0:1789760707.967430:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb463b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3304:x1876699519246848:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.967475:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb463b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3304:x1876699519246848:12345-0@lo:400:kworker.0 Request processed in 46us (2929us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760707.967481:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 449 00000100:00100000:3.0:1789760707.967504:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 450 00000100:00100000:3.0:1789760707.967520:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c3097480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3304:x1876699519246720:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.967548:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c3097480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3304:x1876699519246720:12345-0@lo:400:kworker.0 Request processed in 39us (2949us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760707.967554:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 450 00000100:00100000:3.0:1789760707.967565:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 451 00000100:00100000:3.0:1789760707.967570:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c3095880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3305:x1876699519246464:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.967602:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c3095880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3305:x1876699519246464:12345-0@lo:400:kworker.0 Request processed in 31us (2955us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760707.967607:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 451 00000100:00100000:3.0:1789760707.967634:0:13040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519246976 00000100:00100000:3.0:1789760707.967639:0:13040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519246976 00000100:00100000:3.0:1789760707.967667:0:13040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519246592 00000100:00100000:3.0:1789760707.967669:0:13040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519246592 00000100:00100000:3.0:1789760707.967687:0:13040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 164 00000100:00100000:3.0:1789760707.967694:0:13040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c3097b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519246976:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.967722:0:13040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c3097b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519246976:12345-0@lo:400:kworker.0 Request processed in 29us (3041us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760707.967749:0:13040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 164 00000100:00100000:3.0:1789760707.967760:0:13040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 165 00000100:00100000:3.0:1789760707.967766:0:13040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c3097800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519246592:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.967795:0:13040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c3097800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519246592:12345-0@lo:400:kworker.0 Request processed in 30us (3071us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760707.967800:0:13040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 165 00000100:00100000:3.0:1789760707.968449:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc5837100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3304:1876699519246720:0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.968471:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc5836a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3304:1876699519246336:0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.969281:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620fb460a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3304:1876699519246848:0@lo:400:kworker.0 00000100:00080000:3.0:1789760707.969324:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789760707.969329:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760707.969363:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc5835180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519246592:0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.969376:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc5835500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3305:1876699519246464:0@lo:400:kworker.0 00000100:00100000:3.0:1789760707.969387:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620fb461c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519246976:0@lo:400:kworker.0 00000001:00080000:1.0:1789760707.978000:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972548 is committed 00002000:00010000:0.0:1789760707.980993:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21417984 -> 21422080 00010000:00010000:0.0:1789760707.981014:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04525 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.981027:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04525 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.981036:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04525 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.981042:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760707.981047:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04525 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.981055:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04525 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.981075:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04525 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.981082:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04525 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760707.981375:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8555180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493649280:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47644us (47932us total) trans 4294972548 rc 0/0 00000100:00100000:0.0:1789760707.981389:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5259 00000100:00100000:0.0:1789760707.981399:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718123043915 time 718274749917 state 0x2 ref 0 00000100:00100000:2.0:1789760707.985842:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760707.998596:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493649408 00000100:00100000:1.0:1789760707.998611:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493649408 00000100:00100000:2.0:1789760707.999304:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718123043915 time 718274749917 state 0x0 ref 1 00000100:00100000:2.0:1789760707.999322:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718275010333 time 718274749917 state 0x1 ref 1 00000100:00100000:2.0:1789760707.999355:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718275010333 time 718340391124 state 0x0 ref 1 00000100:00100000:2.0:1789760707.999370:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493649408:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760707.999417:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21422080, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa807 00010000:00010000:2.0:1789760707.999427:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0452c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.999452:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760707.999457:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760707.999463:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0452c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.999476:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0452c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760708.125992:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21422080 -> 21426176 00010000:00010000:1.0:1789760708.126012:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0452c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.126025:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0452c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.126034:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0452c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.126042:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760708.126046:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0452c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.126056:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0452c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.126073:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0452c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.126086:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0452c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760708.126167:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493649408:12345-192.168.203.60@tcp:4:dd.0 Request processed in 126797us (140331us total) trans 4294972549 rc 0/0 00000100:00100000:1.0:1789760708.126177:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5260 00000100:00100000:1.0:1789760708.126187:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718275010333 time 718340391124 state 0x2 ref 0 00000001:00080000:1.0:1789760708.126238:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972549 is committed 00000100:00100000:2.0:1789760708.129650:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760708.129746:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493649536 00000100:00100000:3.0:1789760708.129755:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493649536 00000100:00100000:3.0:1789760708.129915:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718275010333 time 718340391124 state 0x0 ref 1 00000100:00100000:3.0:1789760708.129934:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718340651540 time 718340391124 state 0x1 ref 1 00000100:00100000:3.0:1789760708.130044:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718340651540 time 718471078767 state 0x0 ref 1 00000100:00100000:3.0:1789760708.130059:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493649536:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760708.130107:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21426176, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa807 00010000:00010000:3.0:1789760708.130115:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04533 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.130129:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760708.130134:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760708.130140:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04533 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.130152:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04533 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.158996:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760708.221577:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21426176 -> 21430272 00010000:00010000:3.0:1789760708.221598:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04533 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.221610:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04533 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.221618:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04533 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.221626:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760708.221630:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04533 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.221640:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04533 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.221661:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04533 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.221670:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04533 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760708.221768:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493649536:12345-192.168.203.60@tcp:4:dd.0 Request processed in 91708us (92122us total) trans 4294972550 rc 0/0 00000100:00100000:3.0:1789760708.221789:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5261 00000100:00100000:3.0:1789760708.221799:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718340651540 time 718471078767 state 0x2 ref 0 00000001:00080000:1.0:1789760708.222013:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972550 is committed 00000100:00100000:3.0:1789760708.228688:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760708.228877:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493649664 00000100:00100000:1.0:1789760708.228887:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493649664 00000100:00100000:1.0:1789760708.228943:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718340651540 time 718471078767 state 0x0 ref 1 00000100:00100000:1.0:1789760708.229001:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718471339183 time 718471078767 state 0x1 ref 1 00000100:00100000:1.0:1789760708.229038:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718471339183 time 718570073592 state 0x0 ref 1 00000100:00100000:1.0:1789760708.229052:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493649664:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760708.229113:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21430272, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa807 00010000:00010000:1.0:1789760708.229122:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0453a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.229137:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760708.229142:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760708.229148:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0453a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.229162:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0453a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760708.309697:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972551 is committed 00002000:00010000:0.0:1789760708.309780:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21430272 -> 21434368 00010000:00010000:0.0:1789760708.309796:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0453a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.309810:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0453a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.309819:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0453a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.309827:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760708.309832:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0453a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.309841:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0453a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.309861:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0453a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.309869:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0453a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760708.310045:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493649664:12345-192.168.203.60@tcp:4:dd.0 Request processed in 80992us (81362us total) trans 4294972551 rc 0/0 00000100:00100000:0.0:1789760708.310054:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5262 00000100:00100000:0.0:1789760708.310062:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718471339183 time 718570073592 state 0x2 ref 0 00000100:00100000:2.0:1789760708.316479:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760708.318238:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493649792 00000100:00100000:0.0:1789760708.318250:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493649792 00000100:00100000:0.0:1789760708.318310:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718471339183 time 718570073592 state 0x0 ref 1 00000100:00100000:0.0:1789760708.318325:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718570334008 time 718570073592 state 0x1 ref 1 00000100:00100000:0.0:1789760708.318534:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718570334008 time 718659566717 state 0x0 ref 1 00000100:00100000:0.0:1789760708.318551:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493649792:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760708.318599:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21434368, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa807 00010000:00010000:0.0:1789760708.318606:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04541 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.318621:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760708.318625:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760708.318631:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04541 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.318644:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04541 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760708.393837:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21434368 -> 21438464 00010000:00010000:0.0:1789760708.393856:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04541 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.393870:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04541 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.393879:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04541 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.393886:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760708.393891:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04541 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.393900:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04541 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.393919:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04541 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.393926:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04541 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760708.394041:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493649792:12345-192.168.203.60@tcp:4:dd.0 Request processed in 75490us (77566us total) trans 4294972552 rc 0/0 00000100:00100000:0.0:1789760708.394055:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5263 00000100:00100000:0.0:1789760708.394063:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718570334008 time 718659566717 state 0x2 ref 0 00000001:00080000:2.0:1789760708.395267:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972552 is committed 00000100:00100000:2.0:1789760708.401632:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760708.402390:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493649920 00000100:00100000:3.0:1789760708.402401:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493649920 00000100:00100000:3.0:1789760708.402457:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718570334008 time 718659566717 state 0x0 ref 1 00000100:00100000:3.0:1789760708.402474:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718659827133 time 718659566717 state 0x1 ref 1 00000100:00100000:3.0:1789760708.402498:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718659827133 time 718743533815 state 0x0 ref 1 00000100:00100000:3.0:1789760708.402511:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493649920:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760708.402557:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21438464, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa807 00010000:00010000:3.0:1789760708.402566:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04548 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.402588:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760708.402593:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760708.402599:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04548 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.402614:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04548 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760708.441056:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972553 is committed 00002000:00010000:3.0:1789760708.441163:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21438464 -> 21442560 00010000:00010000:3.0:1789760708.441178:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04548 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.441190:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04548 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.441198:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04548 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.441218:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760708.441222:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04548 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.441231:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04548 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.441251:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04548 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.441268:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04548 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760708.441369:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493649920:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38847us (39734us total) trans 4294972553 rc 0/0 00000100:00100000:3.0:1789760708.441379:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5264 00000100:00100000:3.0:1789760708.441388:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718659827133 time 718743533815 state 0x2 ref 0 00000100:00100000:3.0:1789760708.449277:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760708.449932:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493650048 00000100:00100000:0.0:1789760708.449943:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493650048 00000100:00100000:0.0:1789760708.450044:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718659827133 time 718743533815 state 0x0 ref 1 00000100:00100000:0.0:1789760708.450059:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718743794231 time 718743533815 state 0x1 ref 1 00000100:00100000:0.0:1789760708.450094:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718743794231 time 718791130227 state 0x0 ref 1 00000100:00100000:0.0:1789760708.450107:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493650048:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760708.450157:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21442560, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa807 00010000:00010000:0.0:1789760708.450166:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0454f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.450181:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760708.450185:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760708.450192:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0454f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.450205:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0454f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760708.475093:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4d96a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519247104:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760708.475161:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760708.475484:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519247104 00000100:00100000:2.0:1789760708.475492:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519247104 00000100:00100000:2.0:1789760708.475542:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 187 00000100:00100000:2.0:1789760708.475550:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519247104:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760708.475581:0:6690:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760708.475619:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519247104:12345-0@lo:13:osp-pre-1-0.0 Request processed in 70us (458us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760708.475626:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 187 00000100:00100000:3.0:1789760708.475705:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4d96a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519247104:0@lo:13:osp-pre-1-0.0 00000001:00080000:3.0:1789760708.508270:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972554 is committed 00002000:00010000:2.0:1789760708.508404:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21442560 -> 21446656 00010000:00010000:2.0:1789760708.508420:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0454f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.508435:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0454f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.508451:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0454f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.508459:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760708.508464:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0454f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.508472:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0454f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.508492:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0454f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.508500:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0454f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760708.508595:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493650048:12345-192.168.203.60@tcp:4:dd.0 Request processed in 58485us (59322us total) trans 4294972554 rc 0/0 00000100:00100000:2.0:1789760708.508606:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5265 00000100:00100000:2.0:1789760708.508616:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718743794231 time 718791130227 state 0x2 ref 0 00000100:00100000:2.0:1789760708.514226:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760708.514327:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493650176 00000100:00100000:0.0:1789760708.514336:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493650176 00000100:00100000:0.0:1789760708.514392:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718743794231 time 718791130227 state 0x0 ref 1 00000100:00100000:0.0:1789760708.514408:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718791390643 time 718791130227 state 0x1 ref 1 00000100:00100000:0.0:1789760708.514442:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718791390643 time 718855476962 state 0x0 ref 1 00000100:00100000:0.0:1789760708.514455:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493650176:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760708.514502:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21446656, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa807 00010000:00010000:0.0:1789760708.514510:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04556 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.514526:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760708.514530:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760708.514536:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04556 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.514549:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04556 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760708.607414:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972555 is committed 00002000:00010000:2.0:1789760708.607636:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21446656 -> 21450752 00010000:00010000:2.0:1789760708.607650:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04556 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.607670:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04556 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.607678:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04556 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.607686:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760708.607690:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04556 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.607701:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04556 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.607720:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04556 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.607737:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04556 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760708.607845:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493650176:12345-192.168.203.60@tcp:4:dd.0 Request processed in 93387us (93627us total) trans 4294972555 rc 0/0 00000100:00100000:2.0:1789760708.607855:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5266 00000100:00100000:2.0:1789760708.607866:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718791390643 time 718855476962 state 0x2 ref 0 00000100:00100000:3.0:1789760708.618468:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760708.618647:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493650304 00000100:00100000:1.0:1789760708.618657:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493650304 00000100:00100000:1.0:1789760708.618715:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718791390643 time 718855476962 state 0x0 ref 1 00000100:00100000:1.0:1789760708.618742:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718855737378 time 718855476962 state 0x1 ref 1 00000100:00100000:1.0:1789760708.618775:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718855737378 time 718959810568 state 0x0 ref 1 00000100:00100000:1.0:1789760708.618798:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493650304:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760708.618850:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21450752, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0xa807 00010000:00010000:1.0:1789760708.618859:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0455d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.618875:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760708.618879:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760708.618885:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0455d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760708.618899:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0455d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760708.731276:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972556 is committed 00002000:00010000:3.0:1789760708.731452:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21450752 -> 21454848 00002000:00010000:3.0:1789760708.731456:0:13038:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760707 -> 1789760708 00002000:00010000:3.0:1789760708.731459:0:13038:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760707 -> 1789760708 00010000:00010000:3.0:1789760708.731473:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0455d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.731486:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0455d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.731496:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0455d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.731504:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760708.731508:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0455d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.731517:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0455d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.731537:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0455d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.731546:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0455d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760708.731637:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493650304:12345-192.168.203.60@tcp:4:dd.0 Request processed in 112837us (113174us total) trans 4294972556 rc 0/0 00000100:00100000:3.0:1789760708.731647:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5267 00000100:00100000:3.0:1789760708.731657:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718855737378 time 718959810568 state 0x2 ref 0 00000100:00100000:3.0:1789760708.738582:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760708.738692:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493650432 00000100:00100000:2.0:1789760708.738703:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493650432 00000100:00100000:2.0:1789760708.738768:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718855737378 time 718959810568 state 0x0 ref 1 00000100:00100000:2.0:1789760708.738784:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718960070984 time 718959810568 state 0x1 ref 1 00000100:00100000:2.0:1789760708.738818:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718960070984 time 719079852914 state 0x0 ref 1 00000100:00100000:2.0:1789760708.738831:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493650432:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760708.738882:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21454848, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa807 00010000:00010000:2.0:1789760708.738892:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04564 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.738908:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760708.738912:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760708.738919:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04564 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.738934:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04564 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760708.795634:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21454848 -> 21458944 00010000:00010000:3.0:1789760708.795655:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04564 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.795669:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04564 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.795682:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04564 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.795690:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760708.795694:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04564 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.795703:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04564 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.795720:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04564 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.795745:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04564 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760708.795847:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493650432:12345-192.168.203.60@tcp:4:dd.0 Request processed in 57013us (57270us total) trans 4294972557 rc 0/0 00000100:00100000:3.0:1789760708.795857:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5268 00000100:00100000:3.0:1789760708.795866:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718960070984 time 719079852914 state 0x2 ref 0 00000001:00080000:2.0:1789760708.796213:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972557 is committed 00000100:00100000:2.0:1789760708.805910:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760708.807700:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493650560 00000100:00100000:0.0:1789760708.807715:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493650560 00000100:00100000:0.0:1789760708.807794:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 718960070984 time 719079852914 state 0x0 ref 1 00000100:00100000:0.0:1789760708.807811:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719080113330 time 719079852914 state 0x1 ref 1 00000100:00100000:0.0:1789760708.807847:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719080113330 time 719148882121 state 0x0 ref 1 00000100:00100000:0.0:1789760708.807860:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493650560:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760708.807909:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21458944, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa807 00010000:00010000:0.0:1789760708.807919:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0456b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.807934:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760708.807938:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760708.807945:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0456b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.807994:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0456b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760708.877236:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972558 is committed 00002000:00010000:3.0:1789760708.877411:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21458944 -> 21463040 00010000:00010000:3.0:1789760708.877427:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0456b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.877446:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0456b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.877455:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0456b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.877462:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760708.877467:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0456b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.877477:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0456b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.877496:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0456b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.877504:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0456b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760708.877603:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493650560:12345-192.168.203.60@tcp:4:dd.0 Request processed in 69741us (71698us total) trans 4294972558 rc 0/0 00000100:00100000:3.0:1789760708.877614:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5269 00000100:00100000:3.0:1789760708.877624:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719080113330 time 719148882121 state 0x2 ref 0 00000100:00100000:0.0:1789760708.880888:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760708.881000:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493650688 00000100:00100000:2.0:1789760708.881009:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493650688 00000100:00100000:2.0:1789760708.881066:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719080113330 time 719148882121 state 0x0 ref 1 00000100:00100000:2.0:1789760708.881086:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719149142537 time 719148882121 state 0x1 ref 1 00000100:00100000:2.0:1789760708.881112:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719149142537 time 719222147289 state 0x0 ref 1 00000100:00100000:2.0:1789760708.881125:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb18050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493650688:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760708.881175:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21463040, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa807 00010000:00010000:2.0:1789760708.881185:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04572 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.881200:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760708.881205:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760708.881211:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04572 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.881225:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04572 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760708.922830:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972559 is committed 00002000:00010000:3.0:1789760708.926298:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21463040 -> 21467136 00010000:00010000:3.0:1789760708.926320:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04572 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.926334:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04572 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.926343:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04572 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.926447:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760708.926453:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04572 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.926463:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04572 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.926482:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04572 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.926491:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04572 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760708.926597:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb18050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493650688:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45469us (45713us total) trans 4294972559 rc 0/0 00000100:00100000:3.0:1789760708.926607:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5270 00000100:00100000:3.0:1789760708.926617:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719149142537 time 719222147289 state 0x2 ref 0 00000100:00100000:2.0:1789760708.938081:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760708.938170:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493650816 00000100:00100000:3.0:1789760708.938180:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493650816 00000100:00100000:3.0:1789760708.938243:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719149142537 time 719222147289 state 0x0 ref 1 00000100:00100000:3.0:1789760708.938258:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719222407705 time 719222147289 state 0x1 ref 1 00000100:00100000:3.0:1789760708.938289:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719222407705 time 719279325162 state 0x0 ref 1 00000100:00100000:3.0:1789760708.938302:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493650816:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760708.938349:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21467136, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa807 00010000:00010000:3.0:1789760708.938371:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04579 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.938387:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760708.938391:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760708.938397:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04579 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760708.938412:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04579 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760708.999988:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972560 is committed 00002000:00010000:3.0:1789760709.000232:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21467136 -> 21471232 00010000:00010000:3.0:1789760709.000247:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04579 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.000260:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04579 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.000271:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04579 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.000287:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760709.000293:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04579 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.000304:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04579 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.000326:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04579 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.000334:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04579 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760709.000418:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493650816:12345-192.168.203.60@tcp:4:dd.0 Request processed in 62112us (62342us total) trans 4294972560 rc 0/0 00000100:00100000:3.0:1789760709.000429:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5271 00000100:00100000:3.0:1789760709.000439:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719222407705 time 719279325162 state 0x2 ref 0 00000100:00100000:2.0:1789760709.008260:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760709.008450:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493650944 00000100:00100000:3.0:1789760709.008459:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493650944 00000100:00100000:3.0:1789760709.008530:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719222407705 time 719279325162 state 0x0 ref 1 00000100:00100000:3.0:1789760709.008546:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719279585578 time 719279325162 state 0x1 ref 1 00000100:00100000:3.0:1789760709.008579:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719279585578 time 719349614537 state 0x0 ref 1 00000100:00100000:3.0:1789760709.008592:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493650944:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760709.008643:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21471232, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa807 00010000:00010000:3.0:1789760709.008652:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04580 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.008668:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760709.008673:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760709.008679:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04580 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.008692:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04580 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760709.010201:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760709.093949:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972561 is committed 00002000:00010000:1.0:1789760709.101184:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21471232 -> 21475328 00010000:00010000:1.0:1789760709.101206:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04580 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.101220:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04580 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.101228:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04580 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.101235:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760709.101240:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04580 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.101250:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04580 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.101270:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04580 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.101278:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04580 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760709.101383:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493650944:12345-192.168.203.60@tcp:4:dd.0 Request processed in 92788us (93129us total) trans 4294972561 rc 0/0 00000100:00100000:1.0:1789760709.101394:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5272 00000100:00100000:1.0:1789760709.101403:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719279585578 time 719349614537 state 0x2 ref 0 00000100:00100000:0.0:1789760709.108923:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760709.110424:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493651072 00000100:00100000:3.0:1789760709.110438:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493651072 00000100:00100000:3.0:1789760709.110498:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719279585578 time 719349614537 state 0x0 ref 1 00000100:00100000:3.0:1789760709.110513:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719349874953 time 719349614537 state 0x1 ref 1 00000100:00100000:3.0:1789760709.110553:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719349874953 time 719451588332 state 0x0 ref 1 00000100:00100000:3.0:1789760709.110568:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493651072:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760709.110619:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21475328, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa807 00010000:00010000:3.0:1789760709.110629:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04587 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.110648:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760709.110652:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760709.110658:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04587 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.110672:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04587 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760709.165817:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972562 is committed 00002000:00010000:0.0:1789760709.165903:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21475328 -> 21479424 00010000:00010000:0.0:1789760709.165920:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04587 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.165933:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04587 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.165943:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04587 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.165950:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760709.165956:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04587 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.166052:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04587 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.166074:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04587 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.166301:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04587 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760709.166568:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493651072:12345-192.168.203.60@tcp:4:dd.0 Request processed in 55997us (57647us total) trans 4294972562 rc 0/0 00000100:00100000:1.0:1789760709.166580:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5273 00000100:00100000:1.0:1789760709.166590:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719349874953 time 719451588332 state 0x2 ref 0 00000100:00100000:3.0:1789760709.170643:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760709.170834:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493651200 00000100:00100000:1.0:1789760709.170845:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493651200 00000100:00100000:1.0:1789760709.170898:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719349874953 time 719451588332 state 0x0 ref 1 00000100:00100000:1.0:1789760709.170913:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719451848748 time 719451588332 state 0x1 ref 1 00000100:00100000:1.0:1789760709.170996:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719451848748 time 719512030595 state 0x0 ref 1 00000100:00100000:1.0:1789760709.171008:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493651200:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760709.171057:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21479424, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa807 00010000:00010000:1.0:1789760709.171065:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0458e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.171082:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760709.171086:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760709.171093:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0458e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.171113:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0458e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.178953:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760709.234004:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972563 is committed 00002000:00010000:1.0:1789760709.234113:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21479424 -> 21483520 00010000:00010000:1.0:1789760709.234132:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0458e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.234145:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0458e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.234154:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0458e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.234162:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760709.234167:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0458e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.234176:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0458e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.234199:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0458e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.234206:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0458e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760709.234452:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493651200:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63439us (63812us total) trans 4294972563 rc 0/0 00000100:00100000:1.0:1789760709.234464:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5274 00000100:00100000:1.0:1789760709.234474:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719451848748 time 719512030595 state 0x2 ref 0 00000100:00100000:2.0:1789760709.244474:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760709.246602:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493651328 00000100:00100000:1.0:1789760709.246617:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493651328 00000100:00100000:1.0:1789760709.246685:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719451848748 time 719512030595 state 0x0 ref 1 00000100:00100000:1.0:1789760709.246701:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719512291011 time 719512030595 state 0x1 ref 1 00000100:00100000:1.0:1789760709.246750:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719512291011 time 719587784842 state 0x0 ref 1 00000100:00100000:1.0:1789760709.246764:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493651328:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760709.246819:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21483520, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa807 00010000:00010000:1.0:1789760709.246830:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04595 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.246846:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760709.246850:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760709.246856:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04595 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.246871:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04595 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760709.337930:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972564 is committed 00002000:00010000:1.0:1789760709.338022:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21483520 -> 21487616 00010000:00010000:1.0:1789760709.338038:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04595 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.338052:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04595 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.338062:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04595 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.338070:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760709.338075:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04595 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.338085:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04595 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.338105:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04595 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.338113:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04595 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760709.338196:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493651328:12345-192.168.203.60@tcp:4:dd.0 Request processed in 91430us (93727us total) trans 4294972564 rc 0/0 00000100:00100000:1.0:1789760709.338207:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5275 00000100:00100000:1.0:1789760709.338217:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719512291011 time 719587784842 state 0x2 ref 0 00000100:00100000:1.0:1789760709.343407:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760709.343499:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493651456 00000100:00100000:2.0:1789760709.343509:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493651456 00000100:00100000:2.0:1789760709.343568:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719512291011 time 719587784842 state 0x0 ref 1 00000100:00100000:2.0:1789760709.343582:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719588045258 time 719587784842 state 0x1 ref 1 00000100:00100000:2.0:1789760709.343614:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719588045258 time 719684649632 state 0x0 ref 1 00000100:00100000:2.0:1789760709.343626:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8556a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493651456:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760709.343679:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21487616, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa807 00010000:00010000:2.0:1789760709.343689:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0459c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.343705:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760709.343709:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760709.343716:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0459c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.343743:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0459c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760709.431139:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972565 is committed 00002000:00010000:3.0:1789760709.431217:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21487616 -> 21491712 00010000:00010000:3.0:1789760709.431235:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0459c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.431247:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0459c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.431257:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0459c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.431265:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760709.431269:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0459c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.431284:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0459c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.431303:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0459c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.431311:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0459c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760709.431415:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8556a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493651456:12345-192.168.203.60@tcp:4:dd.0 Request processed in 87786us (88014us total) trans 4294972565 rc 0/0 00000100:00100000:3.0:1789760709.431424:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5276 00000100:00100000:3.0:1789760709.431435:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719588045258 time 719684649632 state 0x2 ref 0 00000100:00100000:3.0:1789760709.435480:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760709.435635:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493651584 00000100:00100000:1.0:1789760709.435646:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493651584 00000100:00100000:1.0:1789760709.435701:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719588045258 time 719684649632 state 0x0 ref 1 00000100:00100000:1.0:1789760709.435728:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719684910048 time 719684649632 state 0x1 ref 1 00000100:00100000:1.0:1789760709.435760:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719684910048 time 719776795218 state 0x0 ref 1 00000100:00100000:1.0:1789760709.435783:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493651584:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760709.435833:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21491712, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa807 00010000:00010000:1.0:1789760709.435841:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045a3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.435857:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760709.435862:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760709.435868:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.435883:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760709.500834:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972566 is committed 00002000:00010000:1.0:1789760709.501121:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21491712 -> 21495808 00010000:00010000:1.0:1789760709.501136:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.501149:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045a3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.501157:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.501164:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760709.501169:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.501178:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045a3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.501207:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045a3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.501216:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045a3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760709.501548:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493651584:12345-192.168.203.60@tcp:4:dd.0 Request processed in 65770us (66072us total) trans 4294972566 rc 0/0 00000100:00100000:1.0:1789760709.501559:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5277 00000100:00100000:1.0:1789760709.501569:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719684910048 time 719776795218 state 0x2 ref 0 00000100:00100000:2.0:1789760709.506872:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760709.506954:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493651712 00000100:00100000:1.0:1789760709.506997:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493651712 00000100:00100000:1.0:1789760709.507051:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719684910048 time 719776795218 state 0x0 ref 1 00000100:00100000:1.0:1789760709.507066:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719777055634 time 719776795218 state 0x1 ref 1 00000100:00100000:1.0:1789760709.507091:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719777055634 time 719848127105 state 0x0 ref 1 00000100:00100000:1.0:1789760709.507104:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493651712:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760709.507155:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21495808, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa807 00010000:00010000:1.0:1789760709.507164:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e045aa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.507180:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760709.507185:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760709.507191:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e045aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.507216:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e045aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760709.568886:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972567 is committed 00002000:00010000:2.0:1789760709.569684:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21495808 -> 21499904 00010000:00010000:2.0:1789760709.569701:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e045aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.569712:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e045aa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.569731:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e045aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.569743:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760709.569748:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e045aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.569757:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e045aa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.569783:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e045aa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.569791:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e045aa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.569886:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493651712:12345-192.168.203.60@tcp:4:dd.0 Request processed in 62779us (63019us total) trans 4294972567 rc 0/0 00000100:00100000:2.0:1789760709.569896:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5278 00000100:00100000:2.0:1789760709.569906:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719777055634 time 719848127105 state 0x2 ref 0 00000100:00100000:0.0:1789760709.578526:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760709.578617:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493651840 00000100:00100000:3.0:1789760709.578628:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493651840 00000100:00100000:3.0:1789760709.578685:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719777055634 time 719848127105 state 0x0 ref 1 00000100:00100000:3.0:1789760709.578701:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719848387521 time 719848127105 state 0x1 ref 1 00000100:00100000:3.0:1789760709.578744:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719848387521 time 719919779572 state 0x0 ref 1 00000100:00100000:3.0:1789760709.578761:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493651840:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760709.578817:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21499904, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0xa807 00010000:00010000:3.0:1789760709.578827:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e045b1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.578843:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760709.578848:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760709.578854:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e045b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.578868:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e045b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760709.620140:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972568 is committed 00002000:00010000:3.0:1789760709.620252:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21499904 -> 21504000 00002000:00010000:3.0:1789760709.620257:0:13038:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760708 -> 1789760709 00002000:00010000:3.0:1789760709.620260:0:13038:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760708 -> 1789760709 00010000:00010000:3.0:1789760709.620272:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e045b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.620410:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e045b1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.620419:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e045b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.620427:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760709.620431:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e045b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.620447:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e045b1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.620466:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e045b1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.620473:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e045b1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760709.620562:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493651840:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41798us (42042us total) trans 4294972568 rc 0/0 00000100:00100000:3.0:1789760709.620572:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5279 00000100:00100000:3.0:1789760709.620581:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719848387521 time 719919779572 state 0x2 ref 0 00000100:00100000:3.0:1789760709.625347:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760709.626204:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493651968 00000100:00100000:2.0:1789760709.626215:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493651968 00000100:00100000:2.0:1789760709.626282:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719848387521 time 719919779572 state 0x0 ref 1 00000100:00100000:2.0:1789760709.626298:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719920039988 time 719919779572 state 0x1 ref 1 00000100:00100000:2.0:1789760709.626330:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719920039988 time 719967365437 state 0x0 ref 1 00000100:00100000:2.0:1789760709.626344:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493651968:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760709.626393:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21504000, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:2.0:1789760709.626402:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e045b8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.626417:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760709.626420:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760709.626426:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e045b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.626439:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e045b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760709.671712:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972569 is committed 00002000:00010000:2.0:1789760709.671850:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21504000 -> 21508096 00010000:00010000:2.0:1789760709.671867:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e045b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.671880:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e045b8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.671890:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e045b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.671898:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760709.671903:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e045b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.671914:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e045b8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.671936:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e045b8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.671945:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e045b8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.672052:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493651968:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45707us (46711us total) trans 4294972569 rc 0/0 00000100:00100000:2.0:1789760709.672062:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5280 00000100:00100000:2.0:1789760709.672073:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719920039988 time 719967365437 state 0x2 ref 0 00000100:00100000:2.0:1789760709.676249:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760709.676372:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493652096 00000100:00100000:1.0:1789760709.676382:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493652096 00000100:00100000:1.0:1789760709.676435:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719920039988 time 719967365437 state 0x0 ref 1 00000100:00100000:1.0:1789760709.676449:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719967625853 time 719967365437 state 0x1 ref 1 00000100:00100000:1.0:1789760709.676479:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719967625853 time 720017515026 state 0x0 ref 1 00000100:00100000:1.0:1789760709.676490:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493652096:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760709.676535:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21508096, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:1.0:1789760709.676544:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045bf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.676559:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760709.676564:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760709.676570:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.676585:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760709.712056:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972570 is committed 00002000:00010000:0.0:1789760709.712217:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21508096 -> 21512192 00010000:00010000:0.0:1789760709.712235:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.712248:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045bf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.712257:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.712266:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760709.712271:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.712286:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045bf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.712306:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045bf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.712314:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045bf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760709.712406:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493652096:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35911us (36164us total) trans 4294972570 rc 0/0 00000100:00100000:0.0:1789760709.712417:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5281 00000100:00100000:0.0:1789760709.712427:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719967625853 time 720017515026 state 0x2 ref 0 00000100:00100000:0.0:1789760709.715790:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760709.717313:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493652224 00000100:00100000:1.0:1789760709.717325:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493652224 00000100:00100000:1.0:1789760709.717387:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 719967625853 time 720017515026 state 0x0 ref 1 00000100:00100000:1.0:1789760709.717404:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720017775442 time 720017515026 state 0x1 ref 1 00000100:00100000:1.0:1789760709.717487:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720017775442 time 720058522002 state 0x0 ref 1 00000100:00100000:1.0:1789760709.717502:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493652224:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760709.717552:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21512192, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:1.0:1789760709.717561:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045c6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.717577:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760709.717581:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760709.717587:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.717601:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760709.767595:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972571 is committed 00002000:00010000:0.0:1789760709.767704:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21512192 -> 21516288 00010000:00010000:0.0:1789760709.767733:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.767747:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045c6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.767757:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.767781:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760709.767786:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.767796:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045c6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.767818:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045c6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.767827:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045c6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760709.767935:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493652224:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50432us (52148us total) trans 4294972571 rc 0/0 00000100:00100000:0.0:1789760709.767945:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5282 00000100:00100000:0.0:1789760709.767956:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720017775442 time 720058522002 state 0x2 ref 0 00000100:00100000:0.0:1789760709.771986:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760709.772023:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493652352 00000100:00100000:0.0:1789760709.772033:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493652352 00000100:00100000:0.0:1789760709.772086:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720017775442 time 720058522002 state 0x0 ref 1 00000100:00100000:0.0:1789760709.772102:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720058782418 time 720058522002 state 0x1 ref 1 00000100:00100000:0.0:1789760709.772127:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720058782418 time 720113162792 state 0x0 ref 1 00000100:00100000:0.0:1789760709.772139:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493652352:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760709.772187:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21516288, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:0.0:1789760709.772265:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e045cd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.772281:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760709.772285:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760709.772291:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e045cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.772306:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e045cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760709.809112:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21516288 -> 21520384 00010000:00010000:0.0:1789760709.809133:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e045cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.809146:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e045cd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.809156:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e045cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.809164:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760709.809169:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e045cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.809179:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e045cd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.809211:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e045cd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.809220:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e045cd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760709.809312:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493652352:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37169us (37353us total) trans 4294972572 rc 0/0 00000100:00100000:0.0:1789760709.809323:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5283 00000100:00100000:0.0:1789760709.809332:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720058782418 time 720113162792 state 0x2 ref 0 00000001:00080000:3.0:1789760709.811315:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972572 is committed 00000100:00100000:2.0:1789760709.814077:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760709.814188:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493652480 00000100:00100000:0.0:1789760709.814341:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493652480 00000100:00100000:1.0:1789760709.814444:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720058782418 time 720113162792 state 0x0 ref 1 00000100:00100000:1.0:1789760709.814460:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720113423208 time 720113162792 state 0x1 ref 1 00000100:00100000:1.0:1789760709.814493:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720113423208 time 720155527910 state 0x0 ref 1 00000100:00100000:1.0:1789760709.814507:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493652480:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760709.814558:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21520384, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:1.0:1789760709.814567:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045d4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.814584:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760709.814588:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760709.814594:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.814613:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760709.863383:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21520384 -> 21524480 00010000:00010000:1.0:1789760709.863404:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.863418:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045d4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.863428:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.863436:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760709.863441:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.863451:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045d4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760709.863466:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972573 is committed 00010000:00010000:1.0:1789760709.863472:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045d4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.863483:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045d4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760709.863572:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493652480:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49063us (49500us total) trans 4294972573 rc 0/0 00000100:00100000:1.0:1789760709.863584:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5284 00000100:00100000:1.0:1789760709.863594:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720113423208 time 720155527910 state 0x2 ref 0 00000100:00100000:2.0:1789760709.870733:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760709.870818:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493652608 00000100:00100000:0.0:1789760709.870828:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493652608 00000100:00100000:0.0:1789760709.870879:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720113423208 time 720155527910 state 0x0 ref 1 00000100:00100000:0.0:1789760709.870894:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720155788326 time 720155527910 state 0x1 ref 1 00000100:00100000:1.0:1789760709.871371:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720155788326 time 720212402814 state 0x0 ref 1 00000100:00100000:1.0:1789760709.871387:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493652608:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760709.871434:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21524480, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:1.0:1789760709.871443:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045db lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.871458:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760709.871462:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760709.871468:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.871483:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760709.890776:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972574 is committed 00002000:00010000:1.0:1789760709.890887:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21524480 -> 21528576 00010000:00010000:1.0:1789760709.890905:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.890914:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045db lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.890928:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.891024:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760709.891028:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.891035:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045db lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.891050:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045db lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.891055:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045db lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760709.891142:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493652608:12345-192.168.203.60@tcp:4:dd.0 Request processed in 19754us (20427us total) trans 4294972574 rc 0/0 00000100:00100000:1.0:1789760709.891162:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5285 00000100:00100000:1.0:1789760709.891172:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720155788326 time 720212402814 state 0x2 ref 0 00000100:00100000:3.0:1789760709.896143:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760709.896940:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493652736 00000100:00100000:0.0:1789760709.896952:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493652736 00000100:00100000:0.0:1789760709.897022:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720155788326 time 720212402814 state 0x0 ref 1 00000100:00100000:0.0:1789760709.897038:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720212663230 time 720212402814 state 0x1 ref 1 00000100:00100000:0.0:1789760709.897071:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720212663230 time 720238106369 state 0x0 ref 1 00000100:00100000:0.0:1789760709.897082:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493652736:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760709.897129:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21528576, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:0.0:1789760709.897137:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e045e2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.897153:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760709.897158:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760709.897164:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e045e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.897180:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e045e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760709.917741:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972575 is committed 00002000:00010000:1.0:1789760709.917764:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21528576 -> 21532672 00010000:00010000:1.0:1789760709.917791:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e045e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.917812:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e045e2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.917822:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e045e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.917830:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760709.917835:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e045e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.917845:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e045e2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.917865:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e045e2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.917873:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e045e2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760709.918021:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493652736:12345-192.168.203.60@tcp:4:dd.0 Request processed in 20933us (21881us total) trans 4294972575 rc 0/0 00000100:00100000:1.0:1789760709.918032:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5286 00000100:00100000:1.0:1789760709.918047:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720212663230 time 720238106369 state 0x2 ref 0 00000100:00100000:2.0:1789760709.920657:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760709.920723:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493652864 00000100:00100000:3.0:1789760709.920733:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493652864 00000100:00100000:3.0:1789760709.920795:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720212663230 time 720238106369 state 0x0 ref 1 00000100:00100000:3.0:1789760709.920807:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720238366785 time 720238106369 state 0x1 ref 1 00000100:00100000:3.0:1789760709.920837:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720238366785 time 720261873034 state 0x0 ref 1 00000100:00100000:3.0:1789760709.920850:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493652864:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760709.920896:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21532672, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:3.0:1789760709.920906:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e045e9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.920921:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760709.920925:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760709.920931:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e045e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.920945:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e045e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760709.941490:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972576 is committed 00002000:00010000:3.0:1789760709.941587:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21532672 -> 21536768 00010000:00010000:3.0:1789760709.942070:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e045e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.942086:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e045e9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.942095:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e045e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.942102:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760709.942107:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e045e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.942117:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e045e9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.942138:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e045e9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760709.942145:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e045e9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760709.942278:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493652864:12345-192.168.203.60@tcp:4:dd.0 Request processed in 21426us (21625us total) trans 4294972576 rc 0/0 00000100:00100000:3.0:1789760709.942290:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5287 00000100:00100000:3.0:1789760709.942300:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720238366785 time 720261873034 state 0x2 ref 0 00000100:00100000:2.0:1789760709.949869:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760709.950288:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493652992 00000100:00100000:1.0:1789760709.950298:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493652992 00000100:00100000:1.0:1789760709.950360:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720238366785 time 720261873034 state 0x0 ref 1 00000100:00100000:1.0:1789760709.950376:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720262133450 time 720261873034 state 0x1 ref 1 00000100:00100000:1.0:1789760709.950447:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720262133450 time 720291481726 state 0x0 ref 1 00000100:00100000:1.0:1789760709.950461:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493652992:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760709.950506:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21536768, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:1.0:1789760709.950514:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045f0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.950530:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760709.950534:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760709.950540:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.950554:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760710.003906:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972577 is committed 00002000:00010000:1.0:1789760710.004140:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21536768 -> 21540864 00010000:00010000:1.0:1789760710.004155:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.004167:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045f0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.004182:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.004189:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760710.004193:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.004269:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045f0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.004288:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045f0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.004294:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e045f0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760710.004428:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493652992:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53964us (54560us total) trans 4294972577 rc 0/0 00000100:00100000:1.0:1789760710.004438:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5288 00000100:00100000:1.0:1789760710.004448:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720262133450 time 720291481726 state 0x2 ref 0 00000100:00100000:0.0:1789760710.009224:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760710.009345:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493653120 00000100:00100000:1.0:1789760710.009356:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493653120 00000100:00100000:1.0:1789760710.009411:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720262133450 time 720291481726 state 0x0 ref 1 00000100:00100000:1.0:1789760710.009426:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720291742142 time 720291481726 state 0x1 ref 1 00000100:00100000:1.0:1789760710.009457:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720291742142 time 720350492894 state 0x0 ref 1 00000100:00100000:1.0:1789760710.009469:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493653120:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760710.009517:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21540864, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:1.0:1789760710.009526:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045f7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.009541:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760710.009545:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760710.009550:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.009560:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760710.084813:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972578 is committed 00002000:00010000:1.0:1789760710.085159:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21540864 -> 21544960 00010000:00010000:1.0:1789760710.085174:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.085187:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045f7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.085196:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.085204:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760710.085209:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.085217:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045f7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.085235:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045f7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.085242:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e045f7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760710.085338:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493653120:12345-192.168.203.60@tcp:4:dd.0 Request processed in 75866us (76119us total) trans 4294972578 rc 0/0 00000100:00100000:1.0:1789760710.085349:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5289 00000100:00100000:1.0:1789760710.085359:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720291742142 time 720350492894 state 0x2 ref 0 00000100:00100000:2.0:1789760710.090679:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760710.090790:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493653248 00000100:00100000:0.0:1789760710.090802:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493653248 00000100:00100000:0.0:1789760710.090859:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720291742142 time 720350492894 state 0x0 ref 1 00000100:00100000:0.0:1789760710.090875:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720350753310 time 720350492894 state 0x1 ref 1 00000100:00100000:0.0:1789760710.090910:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720350753310 time 720431944762 state 0x0 ref 1 00000100:00100000:0.0:1789760710.090923:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493653248:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760710.091009:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21544960, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:0.0:1789760710.091019:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e045fe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.091044:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760710.091049:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760710.091055:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e045fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.091069:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e045fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760710.161780:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972579 is committed 00002000:00010000:0.0:1789760710.161932:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21544960 -> 21549056 00010000:00010000:0.0:1789760710.161947:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e045fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.161987:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e045fe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.161998:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e045fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.162005:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760710.162009:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e045fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.162017:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e045fe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.162043:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e045fe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.162051:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e045fe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760710.162142:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493653248:12345-192.168.203.60@tcp:4:dd.0 Request processed in 71216us (71471us total) trans 4294972579 rc 0/0 00000100:00100000:0.0:1789760710.162152:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5290 00000100:00100000:0.0:1789760710.162162:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720350753310 time 720431944762 state 0x2 ref 0 00000100:00100000:2.0:1789760710.168794:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760710.168916:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493653376 00000100:00100000:3.0:1789760710.168926:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493653376 00000100:00100000:3.0:1789760710.169018:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720350753310 time 720431944762 state 0x0 ref 1 00000100:00100000:3.0:1789760710.169033:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720432205178 time 720431944762 state 0x1 ref 1 00000100:00100000:3.0:1789760710.169062:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720432205178 time 720510097506 state 0x0 ref 1 00000100:00100000:3.0:1789760710.169074:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493653376:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760710.169126:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21549056, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:3.0:1789760710.169134:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04605 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.169150:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760710.169155:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.169161:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04605 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.169175:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04605 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.203915:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760710.255171:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972580 is committed 00002000:00010000:3.0:1789760710.259886:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21549056 -> 21553152 00010000:00010000:3.0:1789760710.259909:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04605 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.259922:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04605 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.259930:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04605 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.259937:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760710.259942:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04605 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.259957:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04605 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.259991:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04605 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.259999:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04605 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760710.260221:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493653376:12345-192.168.203.60@tcp:4:dd.0 Request processed in 91144us (91432us total) trans 4294972580 rc 0/0 00000100:00100000:3.0:1789760710.260232:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5291 00000100:00100000:3.0:1789760710.260241:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720432205178 time 720510097506 state 0x2 ref 0 00000100:00100000:3.0:1789760710.264584:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760710.264684:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493653504 00000100:00100000:1.0:1789760710.264694:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493653504 00000100:00100000:1.0:1789760710.264757:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720432205178 time 720510097506 state 0x0 ref 1 00000100:00100000:1.0:1789760710.264783:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720510357922 time 720510097506 state 0x1 ref 1 00000100:00100000:1.0:1789760710.264811:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720510357922 time 720605847302 state 0x0 ref 1 00000100:00100000:1.0:1789760710.264823:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493653504:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760710.264872:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21553152, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:1.0:1789760710.264880:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0460c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.264907:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760710.264912:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760710.264918:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0460c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.264932:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0460c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760710.351053:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972581 is committed 00002000:00010000:2.0:1789760710.352840:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21553152 -> 21557248 00010000:00010000:2.0:1789760710.352861:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0460c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.352874:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0460c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.352882:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0460c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.352889:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760710.352894:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0460c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.352903:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0460c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.352922:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0460c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.352929:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0460c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760710.353206:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493653504:12345-192.168.203.60@tcp:4:dd.0 Request processed in 88369us (88615us total) trans 4294972581 rc 0/0 00000100:00100000:2.0:1789760710.353217:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5292 00000100:00100000:2.0:1789760710.353227:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720510357922 time 720605847302 state 0x2 ref 0 00000100:00100000:2.0:1789760710.358951:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760710.359119:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493653632 00000100:00100000:1.0:1789760710.359128:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493653632 00000100:00100000:1.0:1789760710.359185:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720510357922 time 720605847302 state 0x0 ref 1 00000100:00100000:1.0:1789760710.359209:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720606107718 time 720605847302 state 0x1 ref 1 00000100:00100000:1.0:1789760710.359283:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720606107718 time 720700317973 state 0x0 ref 1 00000100:00100000:1.0:1789760710.359296:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493653632:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760710.359346:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21557248, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:1.0:1789760710.359354:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04613 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.359369:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760710.359374:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760710.359380:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04613 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.359393:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04613 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760710.449926:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972582 is committed 00002000:00010000:1.0:1789760710.450023:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21557248 -> 21561344 00010000:00010000:1.0:1789760710.450039:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04613 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.450051:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04613 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.450070:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04613 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.450077:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760710.450082:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04613 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.450090:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04613 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.450108:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04613 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.450120:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04613 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760710.450232:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493653632:12345-192.168.203.60@tcp:4:dd.0 Request processed in 90933us (91286us total) trans 4294972582 rc 0/0 00000100:00100000:1.0:1789760710.450242:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5293 00000100:00100000:1.0:1789760710.450252:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720606107718 time 720700317973 state 0x2 ref 0 00000100:00100000:2.0:1789760710.455624:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760710.455736:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493653760 00000100:00100000:0.0:1789760710.455746:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493653760 00000100:00100000:0.0:1789760710.455811:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720606107718 time 720700317973 state 0x0 ref 1 00000100:00100000:0.0:1789760710.455826:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720700578389 time 720700317973 state 0x1 ref 1 00000100:00100000:0.0:1789760710.455857:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720700578389 time 720796892625 state 0x0 ref 1 00000100:00100000:0.0:1789760710.455871:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493653760:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760710.455928:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21561344, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:0.0:1789760710.455937:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0461a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.455953:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760710.455957:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760710.455991:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0461a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.456005:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0461a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760710.512115:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972583 is committed 00002000:00010000:2.0:1789760710.512304:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21561344 -> 21565440 00010000:00010000:2.0:1789760710.512327:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0461a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.512341:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0461a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.512350:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0461a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.512357:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760710.512363:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0461a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.512372:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0461a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.512393:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0461a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.512400:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0461a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760710.512495:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493653760:12345-192.168.203.60@tcp:4:dd.0 Request processed in 56621us (56875us total) trans 4294972583 rc 0/0 00000100:00100000:2.0:1789760710.512506:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5294 00000100:00100000:2.0:1789760710.512516:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720700578389 time 720796892625 state 0x2 ref 0 00000100:00100000:1.0:1789760710.515726:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760710.515861:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493653888 00000100:00100000:3.0:1789760710.515870:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493653888 00000100:00100000:3.0:1789760710.515924:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720700578389 time 720796892625 state 0x0 ref 1 00000100:00100000:3.0:1789760710.515939:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720797153041 time 720796892625 state 0x1 ref 1 00000100:00100000:3.0:1789760710.515997:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720797153041 time 720857031935 state 0x0 ref 1 00000100:00100000:3.0:1789760710.516011:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493653888:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760710.516060:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21565440, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:3.0:1789760710.516111:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04621 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.516128:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760710.516132:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.516138:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04621 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.516151:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04621 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760710.572411:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972584 is committed 00002000:00010000:3.0:1789760710.574917:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21565440 -> 21569536 00010000:00010000:3.0:1789760710.574938:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04621 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.574951:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04621 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.574977:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04621 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.574986:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760710.574991:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04621 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.575000:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04621 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.575021:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04621 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.575029:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04621 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760710.575162:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493653888:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59149us (59451us total) trans 4294972584 rc 0/0 00000100:00100000:3.0:1789760710.575174:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5295 00000100:00100000:3.0:1789760710.575184:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720797153041 time 720857031935 state 0x2 ref 0 00000100:00100000:3.0:1789760710.579698:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760710.580612:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493654016 00000100:00100000:2.0:1789760710.580623:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493654016 00000100:00100000:2.0:1789760710.580681:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720797153041 time 720857031935 state 0x0 ref 1 00000100:00100000:2.0:1789760710.580696:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720857292351 time 720857031935 state 0x1 ref 1 00000100:00100000:2.0:1789760710.580728:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720857292351 time 720921763445 state 0x0 ref 1 00000100:00100000:2.0:1789760710.580740:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493654016:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760710.580802:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21569536, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0xa807 00010000:00010000:2.0:1789760710.580810:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04628 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.580826:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760710.580830:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760710.580836:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04628 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.580848:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04628 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760710.638040:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972585 is committed 00002000:00010000:0.0:1789760710.638362:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21569536 -> 21573632 00002000:00010000:0.0:1789760710.638367:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760709 -> 1789760710 00002000:00010000:0.0:1789760710.638370:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760709 -> 1789760710 00010000:00010000:0.0:1789760710.638383:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04628 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.638394:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04628 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.638403:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04628 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.638411:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760710.638416:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04628 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.638424:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04628 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.638444:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04628 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.638452:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04628 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760710.638557:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493654016:12345-192.168.203.60@tcp:4:dd.0 Request processed in 57814us (58863us total) trans 4294972585 rc 0/0 00000100:00100000:0.0:1789760710.638568:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5296 00000100:00100000:0.0:1789760710.638579:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720857292351 time 720921763445 state 0x2 ref 0 00000100:00100000:2.0:1789760710.646704:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760710.646808:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493654144 00000100:00100000:1.0:1789760710.646817:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493654144 00000100:00100000:1.0:1789760710.646868:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720857292351 time 720921763445 state 0x0 ref 1 00000100:00100000:1.0:1789760710.646882:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720922023861 time 720921763445 state 0x1 ref 1 00000100:00100000:1.0:1789760710.647427:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720922023861 time 720987994750 state 0x0 ref 1 00000100:00100000:1.0:1789760710.647448:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493654144:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760710.647497:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21573632, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:1.0:1789760710.647505:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0462f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.647522:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760710.647527:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760710.647533:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0462f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.647547:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0462f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760710.708895:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972586 is committed 00002000:00010000:1.0:1789760710.709021:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21573632 -> 21577728 00010000:00010000:1.0:1789760710.709036:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0462f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.709049:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0462f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.709069:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0462f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.709077:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760710.709082:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0462f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.709091:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0462f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.709113:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0462f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.709120:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0462f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760710.709217:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493654144:12345-192.168.203.60@tcp:4:dd.0 Request processed in 61771us (62516us total) trans 4294972586 rc 0/0 00000100:00100000:1.0:1789760710.709228:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5297 00000100:00100000:1.0:1789760710.709251:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720922023861 time 720987994750 state 0x2 ref 0 00000100:00100000:1.0:1789760710.712802:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760710.712938:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493654272 00000100:00100000:0.0:1789760710.712948:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493654272 00000100:00100000:0.0:1789760710.713042:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720922023861 time 720987994750 state 0x0 ref 1 00000100:00100000:0.0:1789760710.713057:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720988255166 time 720987994750 state 0x1 ref 1 00000100:00100000:0.0:1789760710.713080:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720988255166 time 721054116810 state 0x0 ref 1 00000100:00100000:0.0:1789760710.713092:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8555500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493654272:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760710.713140:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21577728, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:0.0:1789760710.713149:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04636 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.713164:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760710.713169:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760710.713176:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04636 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.713203:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04636 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760710.772036:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972587 is committed 00002000:00010000:2.0:1789760710.772171:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21577728 -> 21581824 00010000:00010000:2.0:1789760710.772202:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04636 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.772222:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04636 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.772231:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04636 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.772239:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760710.772243:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04636 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.772253:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04636 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.772281:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04636 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.772288:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04636 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760710.772385:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8555500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493654272:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59289us (59589us total) trans 4294972587 rc 0/0 00000100:00100000:2.0:1789760710.772396:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5298 00000100:00100000:2.0:1789760710.772405:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720988255166 time 721054116810 state 0x2 ref 0 00000100:00100000:2.0:1789760710.780373:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760710.780474:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493654400 00000100:00100000:3.0:1789760710.780485:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493654400 00000100:00100000:3.0:1789760710.780543:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 720988255166 time 721054116810 state 0x0 ref 1 00000100:00100000:3.0:1789760710.780558:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721054377226 time 721054116810 state 0x1 ref 1 00000100:00100000:3.0:1789760710.780639:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721054377226 time 721121674521 state 0x0 ref 1 00000100:00100000:3.0:1789760710.780652:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493654400:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760710.780698:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21581824, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:3.0:1789760710.780719:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0463d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.780740:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760710.780745:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760710.780751:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0463d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760710.780875:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0463d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760710.837155:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972588 is committed 00002000:00010000:1.0:1789760710.837568:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21581824 -> 21585920 00010000:00010000:1.0:1789760710.837584:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0463d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.837599:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0463d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.837608:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0463d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.837616:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760710.837622:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0463d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.837630:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0463d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.837650:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0463d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.837664:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0463d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760710.837869:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493654400:12345-192.168.203.60@tcp:4:dd.0 Request processed in 57213us (57550us total) trans 4294972588 rc 0/0 00000100:00100000:1.0:1789760710.837882:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5299 00000100:00100000:1.0:1789760710.837893:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721054377226 time 721121674521 state 0x2 ref 0 00000100:00100000:0.0:1789760710.844936:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760710.845076:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493654528 00000100:00100000:1.0:1789760710.845085:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493654528 00000100:00100000:1.0:1789760710.845149:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721054377226 time 721121674521 state 0x0 ref 1 00000100:00100000:1.0:1789760710.845165:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721121934937 time 721121674521 state 0x1 ref 1 00000100:00100000:1.0:1789760710.845206:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721121934937 time 721186242552 state 0x0 ref 1 00000100:00100000:1.0:1789760710.845220:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493654528:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760710.845287:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21585920, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:1.0:1789760710.845296:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04644 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.845312:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760710.845327:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760710.845333:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04644 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.845353:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04644 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760710.894627:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972589 is committed 00002000:00010000:1.0:1789760710.894720:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21585920 -> 21590016 00010000:00010000:1.0:1789760710.894735:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04644 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.894748:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04644 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.894757:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04644 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.894827:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760710.894832:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04644 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.894842:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04644 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.894863:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04644 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.894870:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04644 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760710.895094:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493654528:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49871us (50163us total) trans 4294972589 rc 0/0 00000100:00100000:1.0:1789760710.895105:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5300 00000100:00100000:1.0:1789760710.895115:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721121934937 time 721186242552 state 0x2 ref 0 00000100:00100000:0.0:1789760710.899866:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760710.904823:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493654656 00000100:00100000:1.0:1789760710.904837:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493654656 00000100:00100000:1.0:1789760710.904900:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721121934937 time 721186242552 state 0x0 ref 1 00000100:00100000:1.0:1789760710.904916:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721186502968 time 721186242552 state 0x1 ref 1 00000100:00100000:1.0:1789760710.905013:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721186502968 time 721246047739 state 0x0 ref 1 00000100:00100000:1.0:1789760710.905027:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493654656:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760710.905079:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21590016, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:1.0:1789760710.905088:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0464b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.905106:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760710.905110:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760710.905117:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0464b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.905132:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0464b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760710.960103:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21590016 -> 21594112 00010000:00010000:0.0:1789760710.960125:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0464b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.960139:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0464b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.960147:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0464b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.960155:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760710.960160:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0464b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.960169:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0464b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.960309:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0464b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.960320:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0464b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760710.960468:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493654656:12345-192.168.203.60@tcp:4:dd.0 Request processed in 55437us (60606us total) trans 4294972590 rc 0/0 00000100:00100000:0.0:1789760710.960480:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5301 00000100:00100000:0.0:1789760710.960491:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721186502968 time 721246047739 state 0x2 ref 0 00000001:00080000:0.0:1789760710.960537:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972590 is committed 00000100:00100000:3.0:1789760710.965508:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760710.965596:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493654784 00000100:00100000:2.0:1789760710.965606:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493654784 00000100:00100000:2.0:1789760710.965662:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721186502968 time 721246047739 state 0x0 ref 1 00000100:00100000:2.0:1789760710.965678:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721246308155 time 721246047739 state 0x1 ref 1 00000100:00100000:2.0:1789760710.965716:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721246308155 time 721306743855 state 0x0 ref 1 00000100:00100000:2.0:1789760710.965729:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493654784:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760710.965790:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21594112, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:2.0:1789760710.965799:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04652 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.965826:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760710.965838:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760710.965845:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04652 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.965861:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04652 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760711.012068:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972591 is committed 00002000:00010000:0.0:1789760711.012173:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21594112 -> 21598208 00010000:00010000:0.0:1789760711.012251:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04652 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.012264:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04652 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.012273:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04652 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.012280:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760711.012285:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04652 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.012294:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04652 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.012318:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04652 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.012325:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04652 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760711.012441:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493654784:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46710us (46939us total) trans 4294972591 rc 0/0 00000100:00100000:0.0:1789760711.012451:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5302 00000100:00100000:0.0:1789760711.012461:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721246308155 time 721306743855 state 0x2 ref 0 00000100:00100000:1.0:1789760711.018150:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760711.020590:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493654912 00000100:00100000:3.0:1789760711.020604:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493654912 00000100:00100000:3.0:1789760711.020724:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721246308155 time 721306743855 state 0x0 ref 1 00000100:00100000:3.0:1789760711.020740:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721307004271 time 721306743855 state 0x1 ref 1 00000100:00100000:3.0:1789760711.020998:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721307004271 time 721362031731 state 0x0 ref 1 00000100:00100000:3.0:1789760711.021014:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8557b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493654912:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760711.021066:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21598208, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:3.0:1789760711.021076:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04659 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.021093:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760711.021098:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.021105:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04659 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.021119:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04659 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760711.022722:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760711.101532:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972592 is committed 00002000:00010000:0.0:1789760711.101727:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21598208 -> 21602304 00010000:00010000:0.0:1789760711.101742:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04659 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.101755:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04659 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.101773:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04659 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.101781:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760711.101786:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04659 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.101801:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04659 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.101821:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04659 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.101829:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04659 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760711.101939:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8557b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493654912:12345-192.168.203.60@tcp:4:dd.0 Request processed in 80923us (83796us total) trans 4294972592 rc 0/0 00000100:00100000:0.0:1789760711.101949:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5303 00000100:00100000:0.0:1789760711.101999:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721307004271 time 721362031731 state 0x2 ref 0 00000100:00100000:0.0:1789760711.106496:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760711.106618:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493655040 00000100:00100000:3.0:1789760711.106627:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493655040 00000100:00100000:3.0:1789760711.106678:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721307004271 time 721362031731 state 0x0 ref 1 00000100:00100000:3.0:1789760711.106692:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721362292147 time 721362031731 state 0x1 ref 1 00000100:00100000:3.0:1789760711.106728:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721362292147 time 721447763636 state 0x0 ref 1 00000100:00100000:3.0:1789760711.106741:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493655040:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760711.106801:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21602304, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:3.0:1789760711.106810:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04660 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.106826:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760711.107036:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.107044:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04660 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.107058:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04660 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760711.142735:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972593 is committed 00002000:00010000:1.0:1789760711.143020:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21602304 -> 21606400 00010000:00010000:1.0:1789760711.143036:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04660 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.143048:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04660 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.143057:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04660 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.143064:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760711.143069:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04660 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.143077:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04660 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.143096:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04660 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.143104:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04660 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.143305:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493655040:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36561us (36812us total) trans 4294972593 rc 0/0 00000100:00100000:1.0:1789760711.143317:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5304 00000100:00100000:1.0:1789760711.143327:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721362292147 time 721447763636 state 0x2 ref 0 00000100:00100000:3.0:1789760711.148883:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760711.149096:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493655168 00000100:00100000:1.0:1789760711.149106:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493655168 00000100:00100000:1.0:1789760711.149160:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721362292147 time 721447763636 state 0x0 ref 1 00000100:00100000:1.0:1789760711.149176:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721448024052 time 721447763636 state 0x1 ref 1 00000100:00100000:1.0:1789760711.149212:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721448024052 time 721490247303 state 0x0 ref 1 00000100:00100000:1.0:1789760711.149226:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493655168:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760711.149277:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21606400, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:1.0:1789760711.149286:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04667 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.149300:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760711.149304:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760711.149310:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04667 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.149325:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04667 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760711.179702:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972594 is committed 00002000:00010000:3.0:1789760711.179820:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21606400 -> 21610496 00010000:00010000:3.0:1789760711.179836:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04667 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.179850:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04667 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.179859:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04667 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.179867:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760711.179872:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04667 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.179881:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04667 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.179900:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04667 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.179907:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04667 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760711.180322:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493655168:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31093us (31443us total) trans 4294972594 rc 0/0 00000100:00100000:3.0:1789760711.180333:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5305 00000100:00100000:3.0:1789760711.180343:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721448024052 time 721490247303 state 0x2 ref 0 00000100:00100000:2.0:1789760711.184414:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760711.185387:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493655296 00000100:00100000:0.0:1789760711.185399:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493655296 00000100:00100000:0.0:1789760711.185538:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721448024052 time 721490247303 state 0x0 ref 1 00000100:00100000:0.0:1789760711.185555:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721490507719 time 721490247303 state 0x1 ref 1 00000100:00100000:0.0:1789760711.185617:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721490507719 time 721526651836 state 0x0 ref 1 00000100:00100000:0.0:1789760711.185630:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493655296:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760711.185678:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21610496, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:0.0:1789760711.185688:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0466e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.185713:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760711.185718:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760711.185724:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0466e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.185738:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0466e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.226268:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760711.248510:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972595 is committed 00002000:00010000:2.0:1789760711.248824:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21610496 -> 21614592 00010000:00010000:2.0:1789760711.248841:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0466e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.248859:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0466e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.248868:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0466e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.248874:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760711.248879:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0466e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.248889:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0466e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.248909:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0466e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.248917:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0466e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760711.249520:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493655296:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63886us (65108us total) trans 4294972595 rc 0/0 00000100:00100000:3.0:1789760711.249531:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5306 00000100:00100000:3.0:1789760711.249541:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721490507719 time 721526651836 state 0x2 ref 0 00000100:00100000:0.0:1789760711.252821:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760711.252932:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493655424 00000100:00100000:3.0:1789760711.252942:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493655424 00000100:00100000:3.0:1789760711.253022:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721490507719 time 721526651836 state 0x0 ref 1 00000100:00100000:3.0:1789760711.253035:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721526912252 time 721526651836 state 0x1 ref 1 00000100:00100000:3.0:1789760711.253057:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721526912252 time 721594093412 state 0x0 ref 1 00000100:00100000:3.0:1789760711.253069:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493655424:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760711.253121:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21614592, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:3.0:1789760711.253130:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04675 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.253145:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760711.253149:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.253154:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04675 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.253172:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04675 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760711.285596:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972596 is committed 00002000:00010000:1.0:1789760711.285713:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21614592 -> 21618688 00010000:00010000:1.0:1789760711.285728:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04675 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.285742:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04675 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.285751:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04675 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.285768:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760711.285772:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04675 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.285781:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04675 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.285800:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04675 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.285806:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04675 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.285900:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493655424:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32828us (33083us total) trans 4294972596 rc 0/0 00000100:00100000:1.0:1789760711.285911:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5307 00000100:00100000:1.0:1789760711.285921:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721526912252 time 721594093412 state 0x2 ref 0 00000100:00100000:3.0:1789760711.289813:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760711.289906:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493655552 00000100:00100000:1.0:1789760711.289915:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493655552 00000100:00100000:1.0:1789760711.290003:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721526912252 time 721594093412 state 0x0 ref 1 00000100:00100000:1.0:1789760711.290019:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721594353828 time 721594093412 state 0x1 ref 1 00000100:00100000:1.0:1789760711.290045:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721594353828 time 721631080678 state 0x0 ref 1 00000100:00100000:1.0:1789760711.290057:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493655552:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760711.290116:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21618688, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:1.0:1789760711.290126:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0467c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.290141:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760711.290146:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760711.290151:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0467c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.290164:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0467c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760711.343079:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972597 is committed 00002000:00010000:2.0:1789760711.343322:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21618688 -> 21622784 00010000:00010000:2.0:1789760711.343340:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0467c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.343354:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0467c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.343363:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0467c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.343371:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760711.343377:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0467c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.343387:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0467c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.343406:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0467c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.343413:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0467c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760711.343506:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493655552:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53446us (53698us total) trans 4294972597 rc 0/0 00000100:00100000:2.0:1789760711.343517:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5308 00000100:00100000:2.0:1789760711.343525:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721594353828 time 721631080678 state 0x2 ref 0 00000100:00100000:0.0:1789760711.347573:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760711.347649:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493655680 00000100:00100000:3.0:1789760711.347657:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493655680 00000100:00100000:3.0:1789760711.347722:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721594353828 time 721631080678 state 0x0 ref 1 00000100:00100000:3.0:1789760711.347737:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721631341094 time 721631080678 state 0x1 ref 1 00000100:00100000:1.0:1789760711.347857:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721631341094 time 721688889633 state 0x0 ref 1 00000100:00100000:1.0:1789760711.347874:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493655680:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760711.347926:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21622784, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:1.0:1789760711.347935:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04683 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.347949:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760711.347954:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760711.347984:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04683 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.348000:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04683 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760711.381719:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972598 is committed 00002000:00010000:2.0:1789760711.381890:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21622784 -> 21626880 00010000:00010000:2.0:1789760711.381906:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04683 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.381925:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04683 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.381934:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04683 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.381942:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760711.381947:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04683 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.381958:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04683 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.382005:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04683 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.382013:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04683 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760711.382171:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493655680:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34297us (34604us total) trans 4294972598 rc 0/0 00000100:00100000:2.0:1789760711.382253:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5309 00000100:00100000:2.0:1789760711.382264:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721631341094 time 721688889633 state 0x2 ref 0 00000100:00100000:3.0:1789760711.386585:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760711.386893:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493655808 00000100:00100000:2.0:1789760711.386905:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493655808 00000100:00100000:2.0:1789760711.387029:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721631341094 time 721688889633 state 0x0 ref 1 00000100:00100000:2.0:1789760711.387047:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721689150049 time 721688889633 state 0x1 ref 1 00000100:00100000:2.0:1789760711.387120:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721689150049 time 721728154601 state 0x0 ref 1 00000100:00100000:2.0:1789760711.387133:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493655808:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760711.387317:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21626880, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:2.0:1789760711.387328:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0468a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.387345:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760711.387349:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760711.387356:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0468a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.387372:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0468a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760711.428322:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21626880 -> 21630976 00010000:00010000:2.0:1789760711.428342:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0468a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.428356:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0468a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.428372:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0468a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.428380:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760711.428386:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0468a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.428396:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0468a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.428415:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0468a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.428423:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0468a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760711.428520:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493655808:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41384us (41940us total) trans 4294972599 rc 0/0 00000100:00100000:2.0:1789760711.428531:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5310 00000100:00100000:2.0:1789760711.428541:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721689150049 time 721728154601 state 0x2 ref 0 00000001:00080000:0.0:1789760711.429102:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972599 is committed 00000100:00100000:3.0:1789760711.432422:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760711.432549:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493655936 00000100:00100000:0.0:1789760711.432560:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493655936 00000100:00100000:0.0:1789760711.432613:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721689150049 time 721728154601 state 0x0 ref 1 00000100:00100000:0.0:1789760711.432627:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721728415017 time 721728154601 state 0x1 ref 1 00000100:00100000:0.0:1789760711.432660:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721728415017 time 721773695194 state 0x0 ref 1 00000100:00100000:0.0:1789760711.432672:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493655936:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760711.432735:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21630976, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:0.0:1789760711.432744:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04691 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.432760:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760711.432764:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760711.432770:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04691 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.432784:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04691 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760711.475638:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21630976 -> 21635072 00010000:00010000:3.0:1789760711.475658:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04691 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.475670:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04691 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.475678:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04691 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.475686:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760711.475691:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04691 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.475716:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04691 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.475735:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04691 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.475742:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e04691 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760711.475906:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493655936:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43230us (43488us total) trans 4294972600 rc 0/0 00000100:00100000:3.0:1789760711.475917:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5311 00000100:00100000:3.0:1789760711.475927:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721728415017 time 721773695194 state 0x2 ref 0 00000001:00080000:1.0:1789760711.476038:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972600 is committed 00000100:00100000:1.0:1789760711.485616:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760711.485895:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493656064 00000100:00100000:2.0:1789760711.485906:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493656064 00000100:00100000:2.0:1789760711.486122:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721728415017 time 721773695194 state 0x0 ref 1 00000100:00100000:2.0:1789760711.486139:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721773955610 time 721773695194 state 0x1 ref 1 00000100:00100000:2.0:1789760711.486174:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721773955610 time 721827209681 state 0x0 ref 1 00000100:00100000:2.0:1789760711.486197:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8555180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493656064:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760711.486248:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21635072, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:2.0:1789760711.486257:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04698 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.486273:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760711.486278:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760711.486285:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04698 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.486298:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04698 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760711.523898:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972601 is committed 00002000:00010000:3.0:1789760711.524170:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21635072 -> 21639168 00010000:00010000:3.0:1789760711.524195:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04698 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.524208:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04698 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.524218:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04698 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.524226:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760711.524231:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04698 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.524240:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04698 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.524262:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04698 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.524270:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04698 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760711.524363:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8555180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493656064:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38164us (38751us total) trans 4294972601 rc 0/0 00000100:00100000:3.0:1789760711.524373:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5312 00000100:00100000:3.0:1789760711.524382:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721773955610 time 721827209681 state 0x2 ref 0 00000100:00100000:0.0:1789760711.529053:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760711.529444:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493656192 00000100:00100000:2.0:1789760711.529455:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493656192 00000100:00100000:2.0:1789760711.529512:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721773955610 time 721827209681 state 0x0 ref 1 00000100:00100000:2.0:1789760711.529528:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721827470097 time 721827209681 state 0x1 ref 1 00000100:00100000:2.0:1789760711.529587:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721827470097 time 721870621140 state 0x0 ref 1 00000100:00100000:2.0:1789760711.529601:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493656192:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760711.529652:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21639168, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:2.0:1789760711.529661:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0469f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.529677:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760711.529681:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760711.529688:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0469f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.529713:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0469f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760711.568674:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972602 is committed 00002000:00010000:3.0:1789760711.568861:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21639168 -> 21643264 00010000:00010000:3.0:1789760711.568876:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0469f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.568890:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0469f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.568899:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0469f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.568911:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760711.568916:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0469f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.568925:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0469f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.568944:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0469f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.568952:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0469f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760711.569085:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493656192:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39482us (40037us total) trans 4294972602 rc 0/0 00000100:00100000:3.0:1789760711.569095:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5313 00000100:00100000:3.0:1789760711.569105:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721827470097 time 721870621140 state 0x2 ref 0 00000100:00100000:1.0:1789760711.573477:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760711.573561:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493656320 00000100:00100000:3.0:1789760711.573571:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493656320 00000100:00100000:3.0:1789760711.573630:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721827470097 time 721870621140 state 0x0 ref 1 00000100:00100000:3.0:1789760711.573644:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721870881556 time 721870621140 state 0x1 ref 1 00000100:00100000:3.0:1789760711.573676:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721870881556 time 721914711375 state 0x0 ref 1 00000100:00100000:3.0:1789760711.573693:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493656320:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760711.573752:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21643264, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0xa807 00010000:00010000:3.0:1789760711.573769:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e046a6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.573785:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760711.573790:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.573795:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e046a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.573810:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e046a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760711.612116:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972603 is committed 00002000:00010000:3.0:1789760711.612227:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21643264 -> 21647360 00002000:00010000:3.0:1789760711.612232:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760710 -> 1789760711 00002000:00010000:3.0:1789760711.612235:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760710 -> 1789760711 00010000:00010000:3.0:1789760711.612247:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e046a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.612259:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e046a6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.612268:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e046a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.612276:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760711.612281:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e046a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.612291:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e046a6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.612311:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e046a6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.612318:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e046a6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760711.612407:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493656320:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38712us (38935us total) trans 4294972603 rc 0/0 00000100:00100000:3.0:1789760711.612417:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5314 00000100:00100000:3.0:1789760711.612427:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721870881556 time 721914711375 state 0x2 ref 0 00000100:00100000:0.0:1789760711.616715:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760711.616834:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493656448 00000100:00100000:3.0:1789760711.616844:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493656448 00000100:00100000:3.0:1789760711.616900:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721870881556 time 721914711375 state 0x0 ref 1 00000100:00100000:3.0:1789760711.616916:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721914971791 time 721914711375 state 0x1 ref 1 00000100:00100000:3.0:1789760711.616947:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721914971791 time 721957982894 state 0x0 ref 1 00000100:00100000:3.0:1789760711.616979:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493656448:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760711.617032:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21647360, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:3.0:1789760711.617041:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046ad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.617055:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760711.617060:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.617066:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.617079:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760711.665642:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972604 is committed 00002000:00010000:3.0:1789760711.666493:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21647360 -> 21651456 00010000:00010000:3.0:1789760711.666510:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.666524:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046ad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.666532:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.666539:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760711.666543:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.666551:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046ad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.666577:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046ad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.666584:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046ad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760711.666727:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493656448:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49764us (50027us total) trans 4294972604 rc 0/0 00000100:00100000:3.0:1789760711.666739:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5315 00000100:00100000:3.0:1789760711.666748:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721914971791 time 721957982894 state 0x2 ref 0 00000100:00100000:2.0:1789760711.672925:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760711.673620:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493656576 00000100:00100000:3.0:1789760711.673632:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493656576 00000100:00100000:3.0:1789760711.673692:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721914971791 time 721957982894 state 0x0 ref 1 00000100:00100000:3.0:1789760711.673718:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721958243310 time 721957982894 state 0x1 ref 1 00000100:00100000:3.0:1789760711.673748:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721958243310 time 722014783618 state 0x0 ref 1 00000100:00100000:3.0:1789760711.673915:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493656576:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760711.674003:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21651456, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:3.0:1789760711.674012:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e046b4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.674029:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760711.674034:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.674041:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e046b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.674054:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e046b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760711.717124:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972605 is committed 00002000:00010000:0.0:1789760711.717350:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21651456 -> 21655552 00010000:00010000:0.0:1789760711.717364:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e046b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.717375:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e046b4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.717383:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e046b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.717391:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760711.717395:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e046b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.717404:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e046b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.717423:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e046b4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.717430:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e046b4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760711.717520:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493656576:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43605us (44600us total) trans 4294972605 rc 0/0 00000100:00100000:0.0:1789760711.717532:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5316 00000100:00100000:0.0:1789760711.717542:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721958243310 time 722014783618 state 0x2 ref 0 00000100:00100000:0.0:1789760711.721146:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760711.721270:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493656704 00000100:00100000:3.0:1789760711.721280:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493656704 00000100:00100000:3.0:1789760711.721334:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 721958243310 time 722014783618 state 0x0 ref 1 00000100:00100000:3.0:1789760711.721348:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722015044034 time 722014783618 state 0x1 ref 1 00000100:00100000:3.0:1789760711.721381:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722015044034 time 722062415119 state 0x0 ref 1 00000100:00100000:3.0:1789760711.721393:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493656704:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760711.721440:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21655552, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:3.0:1789760711.721449:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046bb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.721464:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760711.721468:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.721476:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.721490:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760711.758136:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972606 is committed 00002000:00010000:3.0:1789760711.758363:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21655552 -> 21659648 00010000:00010000:3.0:1789760711.758377:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.758389:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046bb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.758398:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.758405:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760711.758409:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.758418:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046bb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.758435:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046bb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.758442:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046bb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760711.759924:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493656704:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37137us (37393us total) trans 4294972606 rc 0/0 00000100:00100000:3.0:1789760711.759947:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5317 00000100:00100000:3.0:1789760711.759957:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722015044034 time 722062415119 state 0x2 ref 0 00000100:00100000:2.0:1789760711.762768:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760711.762886:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493656832 00000100:00100000:3.0:1789760711.762895:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493656832 00000100:00100000:3.0:1789760711.762975:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722015044034 time 722062415119 state 0x0 ref 1 00000100:00100000:3.0:1789760711.762990:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722062675535 time 722062415119 state 0x1 ref 1 00000100:00100000:3.0:1789760711.763014:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722062675535 time 722104050087 state 0x0 ref 1 00000100:00100000:3.0:1789760711.763025:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493656832:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760711.763073:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21659648, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:3.0:1789760711.763082:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046c2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.763096:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760711.763101:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.763107:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.763120:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760711.784116:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760711.784159:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760711.784201:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760711.784229:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493656960 00000100:00100000:0.0:1789760711.784239:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493656960 00000100:00100000:0.0:1789760711.784285:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 387 00000100:00100000:0.0:1789760711.784295:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d4c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2370:x1876699493656960:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:0.0:1789760711.784366:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d4c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2370:x1876699493656960:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 73us (206us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760711.784373:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 387 00000100:00100000:0.0:1789760711.784403:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493657088 00000100:00100000:0.0:1789760711.784406:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493657088 00000100:00100000:0.0:1789760711.784440:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493657216 00000100:00100000:0.0:1789760711.784443:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493657216 00000100:00100000:0.0:1789760711.784467:0:9460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 452 00000100:00100000:0.0:1789760711.784474:0:9460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2370:x1876699493657088:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:0.0:1789760711.784504:0:9460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2370:x1876699493657088:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 31us (398us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760711.784510:0:9460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 452 00000100:00100000:0.0:1789760711.784522:0:9460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 453 00000100:00100000:0.0:1789760711.784528:0:9460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2370:x1876699493657216:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:0.0:1789760711.784550:0:9460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2370:x1876699493657216:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 22us (348us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760711.784556:0:9460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 453 00000100:00080000:2.0:1789760711.788506:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00002000:00010000:3.0:1789760711.794316:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21659648 -> 21663744 00010000:00010000:3.0:1789760711.794338:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.794351:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046c2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.794360:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.794368:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760711.794374:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.794383:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046c2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.794402:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046c2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.794409:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046c2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760711.794502:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493656832:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31474us (31747us total) trans 4294972607 rc 0/0 00000100:00100000:3.0:1789760711.794513:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5318 00000100:00100000:3.0:1789760711.794523:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722062675535 time 722104050087 state 0x2 ref 0 00000001:00080000:3.0:1789760711.795067:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972607 is committed 00000100:00100000:0.0:1789760711.798531:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760711.799354:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493657344 00000100:00100000:3.0:1789760711.799365:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493657344 00000100:00100000:3.0:1789760711.799424:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722062675535 time 722104050087 state 0x0 ref 1 00000100:00100000:3.0:1789760711.799440:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722104310503 time 722104050087 state 0x1 ref 1 00000100:00100000:3.0:1789760711.799517:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722104310503 time 722140551309 state 0x0 ref 1 00000100:00100000:3.0:1789760711.799530:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493657344:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760711.799579:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21663744, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:3.0:1789760711.799587:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046c9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.799601:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760711.799604:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.799610:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.799623:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760711.826471:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972608 is committed 00002000:00010000:3.0:1789760711.826683:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21663744 -> 21667840 00010000:00010000:3.0:1789760711.826698:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.826720:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.826734:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.826740:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760711.826745:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.826828:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046c9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.826852:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046c9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.826861:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046c9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760711.826991:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493657344:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27427us (28433us total) trans 4294972608 rc 0/0 00000100:00100000:3.0:1789760711.827002:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5319 00000100:00100000:3.0:1789760711.827011:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722104310503 time 722140551309 state 0x2 ref 0 00000100:00100000:0.0:1789760711.830507:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760711.830592:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493657472 00000100:00100000:3.0:1789760711.830603:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493657472 00000100:00100000:3.0:1789760711.830655:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722104310503 time 722140551309 state 0x0 ref 1 00000100:00100000:3.0:1789760711.830669:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722140811725 time 722140551309 state 0x1 ref 1 00000100:00100000:3.0:1789760711.830741:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722140811725 time 722171775787 state 0x0 ref 1 00000100:00100000:3.0:1789760711.830813:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493657472:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760711.830864:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21667840, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:3.0:1789760711.830874:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046d0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.830888:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760711.830893:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.830899:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.830914:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760711.861018:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972609 is committed 00002000:00010000:0.0:1789760711.861115:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21667840 -> 21671936 00010000:00010000:0.0:1789760711.861130:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.861144:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.861152:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.861159:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760711.861163:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.861274:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.861297:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046d0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.861305:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e046d0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760711.861401:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493657472:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30644us (30898us total) trans 4294972609 rc 0/0 00000100:00100000:0.0:1789760711.861413:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5320 00000100:00100000:0.0:1789760711.861423:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722140811725 time 722171775787 state 0x2 ref 0 00000100:00100000:2.0:1789760711.865592:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760711.865684:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493657600 00000100:00100000:0.0:1789760711.865694:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493657600 00000100:00100000:0.0:1789760711.865820:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722140811725 time 722171775787 state 0x0 ref 1 00000100:00100000:0.0:1789760711.865836:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722172036203 time 722171775787 state 0x1 ref 1 00000100:00100000:0.0:1789760711.865903:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722172036203 time 722206938504 state 0x0 ref 1 00000100:00100000:0.0:1789760711.865915:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493657600:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760711.865990:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21671936, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:0.0:1789760711.865998:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e046d7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.866013:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760711.866017:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760711.866023:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e046d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.866036:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e046d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760711.899435:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972610 is committed 00002000:00010000:1.0:1789760711.899532:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21671936 -> 21676032 00010000:00010000:1.0:1789760711.899548:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e046d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.899560:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e046d7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.899569:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e046d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.899576:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760711.899580:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e046d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.899589:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e046d7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.899607:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e046d7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.899614:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e046d7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.899716:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493657600:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33797us (34128us total) trans 4294972610 rc 0/0 00000100:00100000:1.0:1789760711.899727:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5321 00000100:00100000:1.0:1789760711.899736:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722172036203 time 722206938504 state 0x2 ref 0 00000100:00100000:2.0:1789760711.903013:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760711.903126:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493657728 00000100:00100000:0.0:1789760711.903135:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493657728 00000100:00100000:0.0:1789760711.903258:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722172036203 time 722206938504 state 0x0 ref 1 00000100:00100000:0.0:1789760711.903275:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722207198920 time 722206938504 state 0x1 ref 1 00000100:00100000:0.0:1789760711.903302:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722207198920 time 722244337756 state 0x0 ref 1 00000100:00100000:0.0:1789760711.903314:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493657728:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760711.903361:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21676032, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:0.0:1789760711.903370:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e046de lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.903386:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760711.903390:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760711.903396:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e046de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.903415:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e046de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760711.928155:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21676032 -> 21680128 00010000:00010000:3.0:1789760711.928261:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e046de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.928277:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e046de lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.928287:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e046de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.928294:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760711.928300:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e046de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.928309:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e046de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.928329:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e046de lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.928337:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e046de lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760711.928430:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493657728:12345-192.168.203.60@tcp:4:dd.0 Request processed in 25113us (25423us total) trans 4294972611 rc 0/0 00000100:00100000:3.0:1789760711.928440:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5322 00000100:00100000:3.0:1789760711.928451:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722207198920 time 722244337756 state 0x2 ref 0 00000001:00080000:3.0:1789760711.931027:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972611 is committed 00000100:00100000:1.0:1789760711.931527:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760711.931609:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493657856 00000100:00100000:3.0:1789760711.931617:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493657856 00000100:00100000:3.0:1789760711.931667:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722207198920 time 722244337756 state 0x0 ref 1 00000100:00100000:3.0:1789760711.931683:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722244598172 time 722244337756 state 0x1 ref 1 00000100:00100000:3.0:1789760711.931719:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722244598172 time 722272754374 state 0x0 ref 1 00000100:00100000:3.0:1789760711.931732:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493657856:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760711.931785:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21680128, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:3.0:1789760711.931793:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046e5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.931808:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760711.931812:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.931818:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.931831:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760711.978376:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21680128 -> 21684224 00010000:00010000:3.0:1789760711.978395:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.978408:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046e5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.978417:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.978425:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760711.978430:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.978439:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046e5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.978458:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046e5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.978467:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e046e5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760711.978558:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493657856:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46823us (47031us total) trans 4294972612 rc 0/0 00000100:00100000:3.0:1789760711.978570:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5323 00000100:00100000:3.0:1789760711.978581:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722244598172 time 722272754374 state 0x2 ref 0 00000001:00080000:0.0:1789760711.978885:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972612 is committed 00000100:00100000:0.0:1789760711.981922:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760711.982097:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493657984 00000100:00100000:3.0:1789760711.982106:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493657984 00000100:00100000:3.0:1789760711.982161:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722244598172 time 722272754374 state 0x0 ref 1 00000100:00100000:3.0:1789760711.982176:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722273014790 time 722272754374 state 0x1 ref 1 00000100:00100000:3.0:1789760711.982210:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722273014790 time 722323245070 state 0x0 ref 1 00000100:00100000:3.0:1789760711.982222:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493657984:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760711.982271:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21684224, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:3.0:1789760711.982280:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e046ec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.982295:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760711.982301:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760711.982307:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e046ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760711.982322:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e046ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760712.005022:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21684224 -> 21688320 00010000:00010000:3.0:1789760712.005043:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e046ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.005056:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e046ec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.005064:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e046ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.005071:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760712.005076:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e046ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.005085:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e046ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.005104:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e046ec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.005111:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e046ec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.005217:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493657984:12345-192.168.203.60@tcp:4:dd.0 Request processed in 22991us (23299us total) trans 4294972613 rc 0/0 00000100:00100000:3.0:1789760712.005229:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5324 00000100:00100000:3.0:1789760712.005240:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722273014790 time 722323245070 state 0x2 ref 0 00000001:00080000:0.0:1789760712.006851:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972613 is committed 00000100:00100000:0.0:1789760712.009033:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760712.009124:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493658112 00000100:00100000:1.0:1789760712.009134:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493658112 00000100:00100000:1.0:1789760712.009197:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722273014790 time 722323245070 state 0x0 ref 1 00000100:00100000:1.0:1789760712.009212:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722323505486 time 722323245070 state 0x1 ref 1 00000100:00100000:1.0:1789760712.009235:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722323505486 time 722350271080 state 0x0 ref 1 00000100:00100000:1.0:1789760712.009247:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493658112:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760712.009291:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21688320, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:1.0:1789760712.009300:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e046f3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.009316:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760712.009320:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760712.009326:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e046f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.009340:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e046f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760712.039581:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972614 is committed 00002000:00010000:3.0:1789760712.039904:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21688320 -> 21692416 00010000:00010000:3.0:1789760712.039919:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e046f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.039931:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e046f3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.039940:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e046f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.039947:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760712.039952:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e046f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.040050:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e046f3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.040072:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e046f3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.040085:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e046f3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.040377:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493658112:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31125us (31346us total) trans 4294972614 rc 0/0 00000100:00100000:3.0:1789760712.040388:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5325 00000100:00100000:3.0:1789760712.040398:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722323505486 time 722350271080 state 0x2 ref 0 00000100:00100000:1.0:1789760712.048346:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760712.048568:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493658240 00000100:00100000:0.0:1789760712.048578:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493658240 00000100:00100000:0.0:1789760712.048631:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722323505486 time 722350271080 state 0x0 ref 1 00000100:00100000:0.0:1789760712.048645:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722350531496 time 722350271080 state 0x1 ref 1 00000100:00100000:0.0:1789760712.048680:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722350531496 time 722389716035 state 0x0 ref 1 00000100:00100000:0.0:1789760712.048691:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493658240:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760712.048747:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21692416, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:0.0:1789760712.048765:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e046fa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.048779:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760712.048784:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760712.048789:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e046fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.048803:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e046fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760712.094443:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972615 is committed 00002000:00010000:3.0:1789760712.094542:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21692416 -> 21696512 00010000:00010000:3.0:1789760712.094556:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e046fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.094568:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e046fa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.094577:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e046fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.094583:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760712.094588:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e046fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.094596:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e046fa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.094619:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e046fa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.094626:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e046fa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.094843:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493658240:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46148us (46503us total) trans 4294972615 rc 0/0 00000100:00100000:3.0:1789760712.094854:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5326 00000100:00100000:3.0:1789760712.094862:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722350531496 time 722389716035 state 0x2 ref 0 00000100:00100000:0.0:1789760712.098997:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760712.099275:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493658368 00000100:00100000:3.0:1789760712.099285:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493658368 00000100:00100000:3.0:1789760712.099338:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722350531496 time 722389716035 state 0x0 ref 1 00000100:00100000:3.0:1789760712.099353:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722389976451 time 722389716035 state 0x1 ref 1 00000100:00100000:3.0:1789760712.099433:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722389976451 time 722440467214 state 0x0 ref 1 00000100:00100000:3.0:1789760712.099447:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb19850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493658368:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760712.099495:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21696512, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:3.0:1789760712.099505:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04701 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.099520:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760712.099526:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760712.099532:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04701 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.099545:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04701 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760712.130794:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972616 is committed 00002000:00010000:3.0:1789760712.130940:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21696512 -> 21700608 00010000:00010000:3.0:1789760712.130956:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04701 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.131003:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04701 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.131013:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04701 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.131020:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760712.131025:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04701 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.131034:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04701 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.131053:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04701 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.131062:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04701 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.131142:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb19850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493658368:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31693us (32149us total) trans 4294972616 rc 0/0 00000100:00100000:3.0:1789760712.131152:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5327 00000100:00100000:3.0:1789760712.131162:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722389976451 time 722440467214 state 0x2 ref 0 00000100:00100000:0.0:1789760712.136101:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760712.136232:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493658496 00000100:00100000:3.0:1789760712.136242:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493658496 00000100:00100000:3.0:1789760712.136292:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722389976451 time 722440467214 state 0x0 ref 1 00000100:00100000:3.0:1789760712.136306:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722440727630 time 722440467214 state 0x1 ref 1 00000100:00100000:3.0:1789760712.136363:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722440727630 time 722477399252 state 0x0 ref 1 00000100:00100000:3.0:1789760712.136376:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493658496:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760712.136420:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21700608, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:3.0:1789760712.136429:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04708 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.136444:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760712.136448:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760712.136454:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04708 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.136467:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04708 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760712.167431:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972617 is committed 00002000:00010000:0.0:1789760712.167557:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21700608 -> 21704704 00010000:00010000:0.0:1789760712.167573:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04708 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.167587:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04708 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.167596:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04708 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.167605:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760712.167609:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04708 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.167618:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04708 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.167638:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04708 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.167645:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04708 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760712.168110:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493658496:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31728us (32011us total) trans 4294972617 rc 0/0 00000100:00100000:0.0:1789760712.168125:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5328 00000100:00100000:0.0:1789760712.168139:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722440727630 time 722477399252 state 0x2 ref 0 00000100:00100000:1.0:1789760712.174946:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760712.175082:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493658624 00000100:00100000:3.0:1789760712.175093:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493658624 00000100:00100000:3.0:1789760712.175146:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722440727630 time 722477399252 state 0x0 ref 1 00000100:00100000:3.0:1789760712.175161:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722477659668 time 722477399252 state 0x1 ref 1 00000100:00100000:3.0:1789760712.175288:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722477659668 time 722516322211 state 0x0 ref 1 00000100:00100000:3.0:1789760712.175301:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493658624:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760712.175349:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21704704, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:3.0:1789760712.175357:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0470f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.175372:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760712.175376:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760712.175381:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0470f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.175394:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0470f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760712.218576:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21704704 -> 21708800 00010000:00010000:3.0:1789760712.218597:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0470f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.218615:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0470f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.218624:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0470f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.218631:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760712.218635:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0470f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.218643:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0470f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.218673:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0470f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.218681:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0470f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.218783:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493658624:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43480us (43840us total) trans 4294972618 rc 0/0 00000100:00100000:3.0:1789760712.218793:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5329 00000100:00100000:3.0:1789760712.218802:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722477659668 time 722516322211 state 0x2 ref 0 00000001:00080000:2.0:1789760712.219867:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972618 is committed 00000100:00100000:2.0:1789760712.222493:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760712.222570:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493658752 00000100:00100000:0.0:1789760712.222579:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493658752 00000100:00100000:0.0:1789760712.222633:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722477659668 time 722516322211 state 0x0 ref 1 00000100:00100000:0.0:1789760712.222658:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722516582627 time 722516322211 state 0x1 ref 1 00000100:00100000:0.0:1789760712.222686:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722516582627 time 722563722285 state 0x0 ref 1 00000100:00100000:0.0:1789760712.222723:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493658752:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760712.222949:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21708800, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:0.0:1789760712.222982:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04716 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.222998:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760712.223002:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760712.223009:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04716 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.223022:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04716 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.251203:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760712.255534:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21708800 -> 21712896 00010000:00010000:0.0:1789760712.255552:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04716 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.255566:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04716 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.255575:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04716 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.255583:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760712.255589:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04716 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.255598:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04716 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.255618:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04716 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.255625:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04716 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760712.255851:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493658752:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33147us (33361us total) trans 4294972619 rc 0/0 00000100:00100000:0.0:1789760712.255864:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5330 00000100:00100000:0.0:1789760712.255874:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722516582627 time 722563722285 state 0x2 ref 0 00000001:00080000:3.0:1789760712.259946:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972619 is committed 00000100:00100000:0.0:1789760712.260100:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760712.260257:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493658880 00000100:00100000:3.0:1789760712.260265:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493658880 00000100:00100000:3.0:1789760712.260322:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722516582627 time 722563722285 state 0x0 ref 1 00000100:00100000:3.0:1789760712.260337:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722563982701 time 722563722285 state 0x1 ref 1 00000100:00100000:3.0:1789760712.260419:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722563982701 time 722601453734 state 0x0 ref 1 00000100:00100000:3.0:1789760712.260433:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493658880:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760712.260479:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21712896, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:3.0:1789760712.260489:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0471d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.260505:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760712.260510:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760712.260516:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0471d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.260530:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0471d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760712.289290:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972620 is committed 00002000:00010000:2.0:1789760712.289464:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21712896 -> 21716992 00010000:00010000:2.0:1789760712.289481:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0471d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.289494:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0471d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.289502:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0471d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.289510:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760712.289518:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0471d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.289528:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0471d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.289548:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0471d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.289555:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0471d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.289646:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493658880:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29211us (29546us total) trans 4294972620 rc 0/0 00000100:00100000:2.0:1789760712.289736:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5331 00000100:00100000:2.0:1789760712.289746:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722563982701 time 722601453734 state 0x2 ref 0 00000100:00100000:1.0:1789760712.293235:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760712.293368:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493659008 00000100:00100000:3.0:1789760712.293378:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493659008 00000100:00100000:3.0:1789760712.293433:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722563982701 time 722601453734 state 0x0 ref 1 00000100:00100000:3.0:1789760712.293449:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722601714150 time 722601453734 state 0x1 ref 1 00000100:00100000:3.0:1789760712.293484:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722601714150 time 722634520211 state 0x0 ref 1 00000100:00100000:3.0:1789760712.293496:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493659008:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760712.293546:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21716992, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:3.0:1789760712.293554:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04724 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.293568:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760712.293573:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760712.293578:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04724 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.293591:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04724 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760712.324355:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972621 is committed 00002000:00010000:3.0:1789760712.324638:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21716992 -> 21721088 00010000:00010000:3.0:1789760712.324655:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04724 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.324669:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04724 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.324678:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04724 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.324686:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760712.324691:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04724 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.324854:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04724 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.324875:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04724 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.324883:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04724 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.325244:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493659008:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31744us (32016us total) trans 4294972621 rc 0/0 00000100:00100000:3.0:1789760712.325256:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5332 00000100:00100000:3.0:1789760712.325265:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722601714150 time 722634520211 state 0x2 ref 0 00000100:00100000:3.0:1789760712.330756:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760712.330920:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493659136 00000100:00100000:1.0:1789760712.330929:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493659136 00000100:00100000:1.0:1789760712.331016:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722601714150 time 722634520211 state 0x0 ref 1 00000100:00100000:1.0:1789760712.331032:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722634780627 time 722634520211 state 0x1 ref 1 00000100:00100000:1.0:1789760712.331062:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722634780627 time 722672097544 state 0x0 ref 1 00000100:00100000:1.0:1789760712.331075:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493659136:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760712.331124:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21721088, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:1.0:1789760712.331134:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0472b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.331149:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760712.331154:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760712.331160:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0472b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.331174:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0472b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760712.369721:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21721088 -> 21725184 00010000:00010000:2.0:1789760712.369741:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0472b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.369764:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0472b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.369774:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0472b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.369781:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760712.369785:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0472b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.369794:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0472b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.369813:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0472b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.369822:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0472b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.369956:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493659136:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38878us (39203us total) trans 4294972622 rc 0/0 00000100:00100000:2.0:1789760712.370006:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5333 00000100:00100000:2.0:1789760712.370017:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722634780627 time 722672097544 state 0x2 ref 0 00000001:00080000:0.0:1789760712.370215:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972622 is committed 00000100:00100000:0.0:1789760712.376851:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760712.376993:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493659264 00000100:00100000:2.0:1789760712.377005:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493659264 00000100:00100000:2.0:1789760712.377066:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722634780627 time 722672097544 state 0x0 ref 1 00000100:00100000:2.0:1789760712.377082:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722672357960 time 722672097544 state 0x1 ref 1 00000100:00100000:2.0:1789760712.377112:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722672357960 time 722718147640 state 0x0 ref 1 00000100:00100000:2.0:1789760712.377125:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493659264:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760712.377172:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21725184, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:2.0:1789760712.377254:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04732 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.377270:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760712.377275:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760712.377282:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04732 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.377296:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04732 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760712.436095:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972623 is committed 00002000:00010000:2.0:1789760712.436201:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21725184 -> 21729280 00010000:00010000:2.0:1789760712.436217:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04732 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.436231:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04732 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.436241:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04732 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.436248:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760712.436253:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04732 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.436261:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04732 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.436280:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04732 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.436288:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04732 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.436390:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493659264:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59262us (59546us total) trans 4294972623 rc 0/0 00000100:00100000:2.0:1789760712.436400:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5334 00000100:00100000:2.0:1789760712.436410:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722672357960 time 722718147640 state 0x2 ref 0 00000100:00100000:0.0:1789760712.440741:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760712.441137:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493659392 00000100:00100000:2.0:1789760712.441147:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493659392 00000100:00100000:2.0:1789760712.441349:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722672357960 time 722718147640 state 0x0 ref 1 00000100:00100000:2.0:1789760712.441364:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722718408056 time 722718147640 state 0x1 ref 1 00000100:00100000:2.0:1789760712.441459:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722718408056 time 722782492834 state 0x0 ref 1 00000100:00100000:2.0:1789760712.441473:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493659392:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760712.441520:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21729280, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:2.0:1789760712.441529:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04739 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.441544:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760712.441549:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760712.441555:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04739 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.441570:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04739 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760712.477280:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972624 is committed 00002000:00010000:2.0:1789760712.477369:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21729280 -> 21733376 00010000:00010000:2.0:1789760712.477385:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04739 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.477397:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04739 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.477407:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04739 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.477414:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760712.477420:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04739 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.477429:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04739 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.477449:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04739 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.477456:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04739 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.478460:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493659392:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36981us (37720us total) trans 4294972624 rc 0/0 00000100:00100000:2.0:1789760712.478473:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5335 00000100:00100000:2.0:1789760712.478483:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722718408056 time 722782492834 state 0x2 ref 0 00000100:00100000:0.0:1789760712.483548:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760712.483642:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493659520 00000100:00100000:3.0:1789760712.483652:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493659520 00000100:00100000:3.0:1789760712.483859:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722718408056 time 722782492834 state 0x0 ref 1 00000100:00100000:3.0:1789760712.483875:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722782753250 time 722782492834 state 0x1 ref 1 00000100:00100000:3.0:1789760712.483941:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722782753250 time 722824975879 state 0x0 ref 1 00000100:00100000:3.0:1789760712.483956:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493659520:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760712.484095:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21733376, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:3.0:1789760712.484105:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04740 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.484121:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760712.484126:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760712.484133:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04740 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.484148:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04740 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760712.547446:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972625 is committed 00002000:00010000:2.0:1789760712.547726:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21733376 -> 21737472 00010000:00010000:2.0:1789760712.547741:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04740 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.547763:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04740 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.547771:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04740 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.547784:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760712.547789:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04740 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.547799:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04740 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.547817:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04740 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.547824:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04740 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.547918:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493659520:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63960us (64376us total) trans 4294972625 rc 0/0 00000100:00100000:2.0:1789760712.547929:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5336 00000100:00100000:2.0:1789760712.547938:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722782753250 time 722824975879 state 0x2 ref 0 00000100:00100000:1.0:1789760712.558478:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760712.558609:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493659648 00000100:00100000:2.0:1789760712.558619:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493659648 00000100:00100000:2.0:1789760712.558826:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722782753250 time 722824975879 state 0x0 ref 1 00000100:00100000:2.0:1789760712.558843:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722825236295 time 722824975879 state 0x1 ref 1 00000100:00100000:2.0:1789760712.558917:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722825236295 time 722899952296 state 0x0 ref 1 00000100:00100000:2.0:1789760712.558935:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc17480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493659648:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760712.559017:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21737472, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:2.0:1789760712.559026:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04747 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.559043:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760712.559047:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760712.559053:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04747 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.559068:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04747 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.573991:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620ffc15180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3303:1876699519247232:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760712.574080:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760712.574326:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519247232 00000100:00100000:1.0:1789760712.574335:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519247232 00000100:00100000:1.0:1789760712.574395:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 188 00000100:00100000:1.0:1789760712.574404:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876699519247232:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760712.574425:0:6690:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528285696 pending 1703936 free 3836739584 avail 3793747968 00000100:00100000:1.0:1789760712.574468:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876699519247232:12345-0@lo:13:osp-pre-0-0.0 Request processed in 66us (388us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760712.574475:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 188 00000100:00100000:1.0:1789760712.574521:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620ffc15180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3303:1876699519247232:0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1789760712.575528:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789760712.618577:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21737472 -> 21741568 00010000:00010000:2.0:1789760712.618599:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04747 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.618613:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04747 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.618622:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04747 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.618635:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760712.618640:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04747 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.618661:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04747 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.618681:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04747 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.618689:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04747 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.619005:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc17480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493659648:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60027us (60491us total) trans 4294972626 rc 0/0 00000100:00100000:2.0:1789760712.619022:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5337 00000100:00100000:2.0:1789760712.619033:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722825236295 time 722899952296 state 0x2 ref 0 00000001:00080000:0.0:1789760712.619284:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972626 is committed 00000100:00100000:0.0:1789760712.623499:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760712.623619:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493659776 00000100:00100000:2.0:1789760712.623629:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493659776 00000100:00100000:2.0:1789760712.623713:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722825236295 time 722899952296 state 0x0 ref 1 00000100:00100000:2.0:1789760712.623729:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722900212712 time 722899952296 state 0x1 ref 1 00000100:00100000:2.0:1789760712.623768:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722900212712 time 722964803296 state 0x0 ref 1 00000100:00100000:2.0:1789760712.623786:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493659776:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760712.623839:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21741568, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0xa807 00010000:00010000:2.0:1789760712.623848:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0474e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.623863:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760712.623868:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760712.623875:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0474e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.623891:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0474e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760712.677920:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972627 is committed 00002000:00010000:2.0:1789760712.678314:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21741568 -> 21745664 00002000:00010000:2.0:1789760712.678319:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760711 -> 1789760712 00002000:00010000:2.0:1789760712.678322:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760711 -> 1789760712 00010000:00010000:2.0:1789760712.678337:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0474e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.678352:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0474e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.678363:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0474e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.678370:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760712.678375:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0474e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.678383:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0474e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.678409:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0474e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.678418:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0474e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.678611:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493659776:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54822us (55118us total) trans 4294972627 rc 0/0 00000100:00100000:2.0:1789760712.678623:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5338 00000100:00100000:2.0:1789760712.678635:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722900212712 time 722964803296 state 0x2 ref 0 00000100:00100000:1.0:1789760712.682567:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760712.682733:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493659904 00000100:00100000:3.0:1789760712.682806:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493659904 00000100:00100000:3.0:1789760712.682883:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722900212712 time 722964803296 state 0x0 ref 1 00000100:00100000:3.0:1789760712.682899:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722965063712 time 722964803296 state 0x1 ref 1 00000100:00100000:3.0:1789760712.682992:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722965063712 time 723024025830 state 0x0 ref 1 00000100:00100000:3.0:1789760712.683143:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493659904:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760712.683210:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21745664, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:3.0:1789760712.683219:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04755 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.683236:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760712.683242:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760712.683249:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04755 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.683264:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04755 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760712.738585:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972628 is committed 00002000:00010000:2.0:1789760712.738704:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21745664 -> 21749760 00010000:00010000:2.0:1789760712.738720:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04755 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.738732:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04755 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.738741:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04755 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.738826:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760712.738831:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04755 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.738839:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04755 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.738858:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04755 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.738865:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04755 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.739267:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493659904:12345-192.168.203.60@tcp:4:dd.0 Request processed in 56248us (56705us total) trans 4294972628 rc 0/0 00000100:00100000:2.0:1789760712.739278:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5339 00000100:00100000:2.0:1789760712.739288:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722965063712 time 723024025830 state 0x2 ref 0 00000100:00100000:2.0:1789760712.744397:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760712.744502:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493660032 00000100:00100000:3.0:1789760712.744513:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493660032 00000100:00100000:3.0:1789760712.744572:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 722965063712 time 723024025830 state 0x0 ref 1 00000100:00100000:3.0:1789760712.744587:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723024286246 time 723024025830 state 0x1 ref 1 00000100:00100000:3.0:1789760712.744653:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723024286246 time 723085688246 state 0x0 ref 1 00000100:00100000:3.0:1789760712.744666:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493660032:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760712.744727:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21749760, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:3.0:1789760712.744737:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0475c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.744816:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760712.744820:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760712.744827:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0475c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.744841:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0475c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760712.813289:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972629 is committed 00002000:00010000:3.0:1789760712.813378:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21749760 -> 21753856 00010000:00010000:3.0:1789760712.813392:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0475c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.813404:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0475c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.813413:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0475c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.813421:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760712.813426:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0475c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.813436:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0475c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.813455:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0475c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.813463:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0475c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.813592:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493660032:12345-192.168.203.60@tcp:4:dd.0 Request processed in 68922us (69199us total) trans 4294972629 rc 0/0 00000100:00100000:3.0:1789760712.813607:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5340 00000100:00100000:3.0:1789760712.813618:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723024286246 time 723085688246 state 0x2 ref 0 00000100:00100000:0.0:1789760712.817728:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760712.817874:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493660160 00000100:00100000:2.0:1789760712.817884:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493660160 00000100:00100000:2.0:1789760712.817940:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723024286246 time 723085688246 state 0x0 ref 1 00000100:00100000:2.0:1789760712.817955:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723085948662 time 723085688246 state 0x1 ref 1 00000100:00100000:2.0:1789760712.818461:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723085948662 time 723159494199 state 0x0 ref 1 00000100:00100000:2.0:1789760712.818479:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493660160:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760712.818527:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21753856, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:2.0:1789760712.818536:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04763 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.818553:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760712.818557:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760712.818564:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04763 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.818578:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04763 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760712.853250:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21753856 -> 21757952 00010000:00010000:3.0:1789760712.853271:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04763 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.853284:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04763 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.853292:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04763 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.853300:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760712.853304:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04763 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.853313:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04763 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.853331:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04763 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.853339:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04763 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.853434:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493660160:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34955us (35710us total) trans 4294972630 rc 0/0 00000100:00100000:3.0:1789760712.853444:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5341 00000100:00100000:3.0:1789760712.853453:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723085948662 time 723159494199 state 0x2 ref 0 00000001:00080000:0.0:1789760712.856264:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972630 is committed 00000100:00100000:1.0:1789760712.857713:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760712.857836:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493660288 00000100:00100000:2.0:1789760712.857847:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493660288 00000100:00100000:2.0:1789760712.857902:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723085948662 time 723159494199 state 0x0 ref 1 00000100:00100000:2.0:1789760712.857918:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723159754615 time 723159494199 state 0x1 ref 1 00000100:00100000:2.0:1789760712.857948:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723159754615 time 723198983347 state 0x0 ref 1 00000100:00100000:2.0:1789760712.857987:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493660288:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760712.858040:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21757952, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:2.0:1789760712.858049:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0476a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.858069:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760712.858074:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760712.858080:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0476a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.858093:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0476a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760712.885844:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972631 is committed 00002000:00010000:2.0:1789760712.885987:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21757952 -> 21762048 00010000:00010000:2.0:1789760712.886075:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0476a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.886091:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0476a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.886101:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0476a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.886109:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760712.886113:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0476a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.886123:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0476a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.886142:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0476a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.886150:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0476a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.886352:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493660288:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28389us (28641us total) trans 4294972631 rc 0/0 00000100:00100000:2.0:1789760712.886362:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5342 00000100:00100000:2.0:1789760712.886372:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723159754615 time 723198983347 state 0x2 ref 0 00000100:00100000:1.0:1789760712.890609:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760712.890744:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493660416 00000100:00100000:3.0:1789760712.890762:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493660416 00000100:00100000:3.0:1789760712.890822:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723159754615 time 723198983347 state 0x0 ref 1 00000100:00100000:3.0:1789760712.890838:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723199243763 time 723198983347 state 0x1 ref 1 00000100:00100000:3.0:1789760712.890868:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723199243763 time 723231904441 state 0x0 ref 1 00000100:00100000:3.0:1789760712.890880:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493660416:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760712.890927:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21762048, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:3.0:1789760712.890937:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04771 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.890952:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760712.890957:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760712.890984:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04771 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.891183:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04771 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760712.925450:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972632 is committed 00002000:00010000:3.0:1789760712.925549:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21762048 -> 21766144 00010000:00010000:3.0:1789760712.925565:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04771 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.925578:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04771 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.925586:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04771 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.925593:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760712.925598:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04771 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.925607:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04771 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.925626:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04771 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760712.925634:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04771 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760712.925732:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493660416:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34847us (35126us total) trans 4294972632 rc 0/0 00000100:00100000:3.0:1789760712.926149:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5343 00000100:00100000:3.0:1789760712.926160:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723199243763 time 723231904441 state 0x2 ref 0 00000100:00100000:3.0:1789760712.932154:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760712.932425:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493660544 00000100:00100000:2.0:1789760712.932435:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493660544 00000100:00100000:2.0:1789760712.932498:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723199243763 time 723231904441 state 0x0 ref 1 00000100:00100000:2.0:1789760712.932513:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723232164857 time 723231904441 state 0x1 ref 1 00000100:00100000:2.0:1789760712.932599:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723232164857 time 723273633878 state 0x0 ref 1 00000100:00100000:2.0:1789760712.932614:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493660544:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760712.932673:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21766144, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:2.0:1789760712.932683:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04778 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.932707:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760712.932712:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760712.932718:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04778 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.932739:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04778 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760712.969741:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21766144 -> 21770240 00010000:00010000:2.0:1789760712.969777:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04778 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.969911:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04778 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.969920:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04778 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.969928:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760712.969933:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04778 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.969943:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04778 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.969985:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04778 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.969994:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04778 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.970089:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493660544:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37474us (37941us total) trans 4294972633 rc 0/0 00000100:00100000:2.0:1789760712.970101:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5344 00000100:00100000:2.0:1789760712.970111:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723232164857 time 723273633878 state 0x2 ref 0 00000001:00080000:0.0:1789760712.971123:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972633 is committed 00000100:00100000:1.0:1789760712.974747:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760712.974893:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493660672 00000100:00100000:2.0:1789760712.974903:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493660672 00000100:00100000:2.0:1789760712.975013:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723232164857 time 723273633878 state 0x0 ref 1 00000100:00100000:2.0:1789760712.975029:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723273894294 time 723273633878 state 0x1 ref 1 00000100:00100000:2.0:1789760712.975053:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723273894294 time 723316088762 state 0x0 ref 1 00000100:00100000:2.0:1789760712.975066:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493660672:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760712.975117:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21770240, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:2.0:1789760712.975126:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0477f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.975142:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760712.975147:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760712.975154:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0477f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.975168:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0477f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760713.009588:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21770240 -> 21774336 00010000:00010000:0.0:1789760713.009610:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0477f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.009623:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0477f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.009633:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0477f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.009646:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760713.009652:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0477f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.009662:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0477f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.009683:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0477f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.009699:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0477f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760713.009810:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493660672:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34741us (35066us total) trans 4294972634 rc 0/0 00000100:00100000:0.0:1789760713.009821:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5345 00000100:00100000:0.0:1789760713.009830:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723273894294 time 723316088762 state 0x2 ref 0 00000001:00080000:1.0:1789760713.013461:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972634 is committed 00000100:00100000:2.0:1789760713.016710:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760713.016931:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493660800 00000100:00100000:0.0:1789760713.016940:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493660800 00000100:00100000:0.0:1789760713.017038:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723273894294 time 723316088762 state 0x0 ref 1 00000100:00100000:0.0:1789760713.017054:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723316349178 time 723316088762 state 0x1 ref 1 00000100:00100000:0.0:1789760713.017087:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723316349178 time 723358122402 state 0x0 ref 1 00000100:00100000:0.0:1789760713.017103:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493660800:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760713.017152:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21774336, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:0.0:1789760713.017160:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04786 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.017176:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760713.017181:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760713.017187:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04786 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.017395:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04786 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760713.018390:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760713.050162:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972635 is committed 00002000:00010000:2.0:1789760713.050297:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21774336 -> 21778432 00010000:00010000:2.0:1789760713.050313:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04786 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.050327:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04786 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.050336:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04786 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.050344:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760713.050348:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04786 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.050358:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04786 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.050377:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04786 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.050384:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04786 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760713.050485:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493660800:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33379us (33791us total) trans 4294972635 rc 0/0 00000100:00100000:2.0:1789760713.050497:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5346 00000100:00100000:2.0:1789760713.050508:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723316349178 time 723358122402 state 0x2 ref 0 00000100:00100000:2.0:1789760713.055019:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760713.055163:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493660928 00000100:00100000:0.0:1789760713.055173:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493660928 00000100:00100000:0.0:1789760713.055231:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723316349178 time 723358122402 state 0x0 ref 1 00000100:00100000:0.0:1789760713.055246:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723358382818 time 723358122402 state 0x1 ref 1 00000100:00100000:0.0:1789760713.055280:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723358382818 time 723396314855 state 0x0 ref 1 00000100:00100000:0.0:1789760713.055293:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493660928:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760713.055341:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21778432, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:0.0:1789760713.055350:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0478d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.055365:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760713.055369:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760713.055375:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0478d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.055388:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0478d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760713.083684:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4efbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3304:1876699519247360:0@lo:400:kworker.0 00000100:00100000:2.0:1789760713.083762:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760713.083822:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4efaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3304:1876699519247872:0@lo:400:kworker.0 00000100:00100000:2.0:1789760713.083846:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760713.083869:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760713.083875:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4efb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3304:1876699519247744:0@lo:400:kworker.0 00000100:00100000:2.0:1789760713.083894:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760713.083989:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519247360 00000100:00100000:1.0:1789760713.084007:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519247360 00000100:00100000:2.0:1789760713.084048:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4ef8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3305:1876699519247488:0@lo:400:kworker.0 00000100:00100000:1.0:1789760713.084049:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 388 00000100:00100000:1.0:1789760713.084058:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d4f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3304:x1876699519247360:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760713.084075:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760713.084095:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4efa300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519248000:0@lo:400:kworker.0 00000100:00100000:1.0:1789760713.084108:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d4f850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3304:x1876699519247360:12345-0@lo:400:kworker.0 Request processed in 51us (346us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760713.084115:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 388 00000100:00100000:2.0:1789760713.084116:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760713.084133:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789760713.084139:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4ef8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519247616:0@lo:400:kworker.0 00000100:00100000:2.0:1789760713.084163:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760713.084247:0:13040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519248000 00000100:00100000:1.0:1789760713.084252:0:13040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519248000 00000100:00100000:2.0:1789760713.084275:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519247872 00000100:00100000:2.0:1789760713.084279:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519247872 00000100:00100000:1.0:1789760713.084295:0:13040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519247616 00000100:00100000:1.0:1789760713.084298:0:13040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519247616 00000100:00100000:2.0:1789760713.084305:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519247744 00000100:00100000:2.0:1789760713.084307:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519247744 00000100:00100000:1.0:1789760713.084317:0:13040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 166 00000100:00100000:1.0:1789760713.084323:0:13040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fadb7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519248000:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760713.084324:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519247488 00000100:00100000:2.0:1789760713.084326:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519247488 00000100:00100000:2.0:1789760713.084338:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 454 00000100:00100000:2.0:1789760713.084344:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3304:x1876699519247872:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760713.084359:0:13040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fadb7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519248000:12345-0@lo:400:kworker.0 Request processed in 36us (242us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760713.084365:0:13040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 166 00000100:00100000:2.0:1789760713.084372:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3304:x1876699519247872:12345-0@lo:400:kworker.0 Request processed in 29us (525us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760713.084376:0:13040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 167 00000100:00100000:2.0:1789760713.084378:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 454 00000100:00100000:1.0:1789760713.084382:0:13040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fadb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519247616:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760713.084390:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 455 00000100:00100000:2.0:1789760713.084395:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3304:x1876699519247744:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760713.084406:0:13040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fadb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519247616:12345-0@lo:400:kworker.0 Request processed in 24us (242us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760713.084411:0:13040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 167 00000100:00100000:2.0:1789760713.084432:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3304:x1876699519247744:12345-0@lo:400:kworker.0 Request processed in 36us (536us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760713.084438:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 455 00000100:00100000:2.0:1789760713.084450:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 456 00000100:00100000:2.0:1789760713.084456:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620f39f5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3305:x1876699519247488:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760713.084484:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620f39f5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3305:x1876699519247488:12345-0@lo:400:kworker.0 Request processed in 28us (408us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760713.084490:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 456 00000100:00080000:3.0:1789760713.084501:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789760713.084507:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789760713.084546:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4ef8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519247616:0@lo:400:kworker.0 00000100:00100000:2.0:1789760713.084555:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4ef8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3305:1876699519247488:0@lo:400:kworker.0 00000100:00100000:2.0:1789760713.084564:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4efa300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519248000:0@lo:400:kworker.0 00000100:00100000:2.0:1789760713.084595:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4efb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3304:1876699519247744:0@lo:400:kworker.0 00000100:00100000:2.0:1789760713.084603:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4efbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3304:1876699519247360:0@lo:400:kworker.0 00000100:00100000:2.0:1789760713.084612:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4efaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3304:1876699519247872:0@lo:400:kworker.0 00000001:00080000:1.0:1789760713.123444:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972636 is committed 00002000:00010000:0.0:1789760713.123550:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21778432 -> 21782528 00010000:00010000:0.0:1789760713.123565:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0478d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.123579:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0478d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.123588:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0478d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.123597:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760713.123602:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0478d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.123611:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0478d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.123629:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0478d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.123638:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0478d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760713.123874:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493660928:12345-192.168.203.60@tcp:4:dd.0 Request processed in 68579us (68918us total) trans 4294972636 rc 0/0 00000100:00100000:0.0:1789760713.123884:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5347 00000100:00100000:0.0:1789760713.123894:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723358382818 time 723396314855 state 0x2 ref 0 00000100:00100000:0.0:1789760713.136384:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760713.136481:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493661056 00000100:00100000:3.0:1789760713.136490:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493661056 00000100:00100000:3.0:1789760713.136543:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723358382818 time 723396314855 state 0x0 ref 1 00000100:00100000:3.0:1789760713.136560:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723396575271 time 723396314855 state 0x1 ref 1 00000100:00100000:3.0:1789760713.136591:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723396575271 time 723477626832 state 0x0 ref 1 00000100:00100000:3.0:1789760713.136603:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493661056:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760713.136655:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21782528, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:3.0:1789760713.136664:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04794 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.136679:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760713.136684:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.136701:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04794 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.136717:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04794 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760713.190857:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972637 is committed 00002000:00010000:0.0:1789760713.191347:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21782528 -> 21786624 00010000:00010000:0.0:1789760713.191365:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04794 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.191379:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04794 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.191388:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04794 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.191395:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760713.191400:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04794 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.191408:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04794 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.191431:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04794 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.191439:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04794 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760713.191534:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493661056:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54927us (55154us total) trans 4294972637 rc 0/0 00000100:00100000:0.0:1789760713.191545:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5348 00000100:00100000:0.0:1789760713.191555:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723396575271 time 723477626832 state 0x2 ref 0 00000100:00100000:1.0:1789760713.196580:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760713.196661:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493661184 00000100:00100000:2.0:1789760713.196671:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493661184 00000100:00100000:2.0:1789760713.196738:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723396575271 time 723477626832 state 0x0 ref 1 00000100:00100000:2.0:1789760713.196892:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723477887248 time 723477626832 state 0x1 ref 1 00000100:00100000:2.0:1789760713.196979:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723477887248 time 723537994421 state 0x0 ref 1 00000100:00100000:2.0:1789760713.196994:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493661184:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760713.197048:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21786624, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:2.0:1789760713.197057:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0479b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.197073:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760713.197078:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760713.197085:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0479b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.197099:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0479b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760713.228932:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972638 is committed 00002000:00010000:2.0:1789760713.230634:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21786624 -> 21790720 00010000:00010000:2.0:1789760713.230654:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0479b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.230668:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0479b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.230676:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0479b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.230698:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760713.230703:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0479b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.230711:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0479b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.230731:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0479b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.230738:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0479b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760713.231035:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493661184:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34038us (34460us total) trans 4294972638 rc 0/0 00000100:00100000:2.0:1789760713.231045:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5349 00000100:00100000:2.0:1789760713.231054:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723477887248 time 723537994421 state 0x2 ref 0 00000100:00100000:1.0:1789760713.238675:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760713.238876:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493661312 00000100:00100000:2.0:1789760713.238886:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493661312 00000100:00100000:2.0:1789760713.238944:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723477887248 time 723537994421 state 0x0 ref 1 00000100:00100000:2.0:1789760713.239033:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723538254837 time 723537994421 state 0x1 ref 1 00000100:00100000:2.0:1789760713.239092:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723538254837 time 723580127329 state 0x0 ref 1 00000100:00100000:2.0:1789760713.239109:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc352d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493661312:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760713.239159:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21790720, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:2.0:1789760713.239177:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e047a2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.239199:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760713.239204:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760713.239210:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e047a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.239222:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e047a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.274117:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760713.274677:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972639 is committed 00002000:00010000:3.0:1789760713.274802:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21790720 -> 21794816 00010000:00010000:3.0:1789760713.274821:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e047a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.274836:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e047a2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.274844:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e047a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.274851:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760713.274855:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e047a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.274863:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e047a2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.274881:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e047a2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.274889:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e047a2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.275064:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493661312:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35952us (36394us total) trans 4294972639 rc 0/0 00000100:00100000:3.0:1789760713.275078:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5350 00000100:00100000:3.0:1789760713.275088:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723538254837 time 723580127329 state 0x2 ref 0 00000100:00100000:0.0:1789760713.280162:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760713.280441:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493661440 00000100:00100000:3.0:1789760713.280452:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493661440 00000100:00100000:3.0:1789760713.280507:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723538254837 time 723580127329 state 0x0 ref 1 00000100:00100000:3.0:1789760713.280523:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723580387745 time 723580127329 state 0x1 ref 1 00000100:00100000:3.0:1789760713.280556:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723580387745 time 723621591249 state 0x0 ref 1 00000100:00100000:3.0:1789760713.280569:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493661440:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760713.280620:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21794816, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:3.0:1789760713.280629:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047a9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.280645:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760713.280649:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.280655:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.280670:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760713.314166:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972640 is committed 00002000:00010000:3.0:1789760713.314398:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21794816 -> 21798912 00010000:00010000:3.0:1789760713.314412:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.314426:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047a9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.314435:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.314441:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760713.314446:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.314454:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047a9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.314472:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047a9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.314479:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047a9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.314600:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493661440:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34029us (34444us total) trans 4294972640 rc 0/0 00000100:00100000:3.0:1789760713.314611:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5351 00000100:00100000:3.0:1789760713.314620:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723580387745 time 723621591249 state 0x2 ref 0 00000100:00100000:2.0:1789760713.318906:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760713.319053:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493661568 00000100:00100000:0.0:1789760713.319062:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493661568 00000100:00100000:0.0:1789760713.319119:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723580387745 time 723621591249 state 0x0 ref 1 00000100:00100000:0.0:1789760713.319136:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723621851665 time 723621591249 state 0x1 ref 1 00000100:00100000:0.0:1789760713.319303:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723621851665 time 723660337293 state 0x0 ref 1 00000100:00100000:0.0:1789760713.319317:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493661568:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760713.319364:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21798912, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:0.0:1789760713.319374:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e047b0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.319394:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760713.319399:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760713.319405:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e047b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.319418:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e047b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760713.350881:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972641 is committed 00002000:00010000:0.0:1789760713.351096:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21798912 -> 21803008 00010000:00010000:0.0:1789760713.351111:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e047b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.351125:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e047b0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.351134:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e047b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.351142:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760713.351147:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e047b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.351156:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e047b0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.351186:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e047b0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.351296:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e047b0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760713.351379:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493661568:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32059us (32477us total) trans 4294972641 rc 0/0 00000100:00100000:0.0:1789760713.351390:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5352 00000100:00100000:0.0:1789760713.351401:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723621851665 time 723660337293 state 0x2 ref 0 00000100:00100000:1.0:1789760713.358532:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760713.358627:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493661696 00000100:00100000:2.0:1789760713.358637:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493661696 00000100:00100000:2.0:1789760713.358704:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723621851665 time 723660337293 state 0x0 ref 1 00000100:00100000:2.0:1789760713.358721:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723660597709 time 723660337293 state 0x1 ref 1 00000100:00100000:3.0:1789760713.360051:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723660597709 time 723701079422 state 0x0 ref 1 00000100:00100000:3.0:1789760713.360087:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc352c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493661696:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760713.360142:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21803008, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:3.0:1789760713.360152:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047b7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.360177:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760713.360182:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.360257:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.360273:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760713.412340:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972642 is committed 00002000:00010000:3.0:1789760713.414054:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21803008 -> 21807104 00010000:00010000:3.0:1789760713.414085:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.414097:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047b7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.414105:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.414111:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760713.414116:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.414125:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047b7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.414143:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047b7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.414150:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047b7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.414484:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493661696:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54400us (55955us total) trans 4294972642 rc 0/0 00000100:00100000:3.0:1789760713.414494:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5353 00000100:00100000:3.0:1789760713.414503:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723660597709 time 723701079422 state 0x2 ref 0 00000100:00100000:0.0:1789760713.421243:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760713.421367:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493661824 00000100:00100000:3.0:1789760713.421376:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493661824 00000100:00100000:3.0:1789760713.421427:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723660597709 time 723701079422 state 0x0 ref 1 00000100:00100000:3.0:1789760713.421440:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723701339838 time 723701079422 state 0x1 ref 1 00000100:00100000:3.0:1789760713.421469:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723701339838 time 723762504745 state 0x0 ref 1 00000100:00100000:3.0:1789760713.421480:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493661824:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760713.421527:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21807104, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:3.0:1789760713.421536:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e047be lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.421552:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760713.421556:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.421562:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e047be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.421577:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e047be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760713.479541:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972643 is committed 00002000:00010000:2.0:1789760713.479633:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21807104 -> 21811200 00010000:00010000:2.0:1789760713.479650:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e047be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.479669:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e047be lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.479679:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e047be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.479693:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760713.479698:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e047be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.479708:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e047be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.479728:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e047be lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.479735:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e047be lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760713.480023:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493661824:12345-192.168.203.60@tcp:4:dd.0 Request processed in 58537us (58782us total) trans 4294972643 rc 0/0 00000100:00100000:2.0:1789760713.480035:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5354 00000100:00100000:2.0:1789760713.480046:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723701339838 time 723762504745 state 0x2 ref 0 00000100:00100000:2.0:1789760713.484318:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760713.484929:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493661952 00000100:00100000:0.0:1789760713.484939:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493661952 00000100:00100000:0.0:1789760713.485092:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723701339838 time 723762504745 state 0x0 ref 1 00000100:00100000:0.0:1789760713.485109:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723762765161 time 723762504745 state 0x1 ref 1 00000100:00100000:2.0:1789760713.485334:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723762765161 time 723826367154 state 0x0 ref 1 00000100:00100000:2.0:1789760713.485352:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493661952:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760713.485408:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21811200, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:2.0:1789760713.485417:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e047c5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.485436:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760713.485440:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760713.485447:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e047c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.485461:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e047c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760713.546716:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972644 is committed 00002000:00010000:3.0:1789760713.547050:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21811200 -> 21815296 00010000:00010000:3.0:1789760713.547065:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e047c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.547078:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e047c5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.547087:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e047c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.547095:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760713.547100:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e047c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.547108:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e047c5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.547127:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e047c5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.547135:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e047c5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.547424:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493661952:12345-192.168.203.60@tcp:4:dd.0 Request processed in 62071us (63108us total) trans 4294972644 rc 0/0 00000100:00100000:3.0:1789760713.547434:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5355 00000100:00100000:3.0:1789760713.547445:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723762765161 time 723826367154 state 0x2 ref 0 00000100:00100000:1.0:1789760713.551586:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760713.551713:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493662080 00000100:00100000:0.0:1789760713.551723:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493662080 00000100:00100000:0.0:1789760713.551900:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723762765161 time 723826367154 state 0x0 ref 1 00000100:00100000:0.0:1789760713.551917:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723826627570 time 723826367154 state 0x1 ref 1 00000100:00100000:2.0:1789760713.552117:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723826627570 time 723893149059 state 0x0 ref 1 00000100:00100000:2.0:1789760713.552134:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc352f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493662080:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760713.552362:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21815296, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:2.0:1789760713.552372:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e047cc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.552387:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760713.552392:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760713.552399:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e047cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.552419:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e047cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760713.586073:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21815296 -> 21819392 00010000:00010000:3.0:1789760713.586092:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e047cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.586106:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e047cc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.586114:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e047cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.586122:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760713.586126:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e047cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.586135:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e047cc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.586158:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e047cc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.586175:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e047cc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.586328:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493662080:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34194us (34749us total) trans 4294972645 rc 0/0 00000100:00100000:3.0:1789760713.586338:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5356 00000100:00100000:3.0:1789760713.586348:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723826627570 time 723893149059 state 0x2 ref 0 00000100:00100000:0.0:1789760713.591075:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760713.591512:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493662208 00000100:00100000:3.0:1789760713.591523:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493662208 00000100:00100000:3.0:1789760713.591581:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723826627570 time 723893149059 state 0x0 ref 1 00000100:00100000:3.0:1789760713.591597:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723893409475 time 723893149059 state 0x1 ref 1 00000100:00100000:3.0:1789760713.591634:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723893409475 time 723932670026 state 0x0 ref 1 00000100:00100000:3.0:1789760713.591646:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493662208:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760713.591707:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21819392, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0xa807 00010000:00010000:3.0:1789760713.591717:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047d3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.591732:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760713.591738:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.591744:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.591992:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760713.592335:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972645 is committed 00000100:00100000:2.0:1789760713.597883:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4efb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519248128:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760713.597956:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760713.598043:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519248128 00000100:00100000:2.0:1789760713.598051:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519248128 00000100:00100000:2.0:1789760713.598097:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 189 00000100:00100000:2.0:1789760713.598105:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519248128:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760713.598136:0:6690:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760713.598313:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519248128:12345-0@lo:13:osp-pre-1-0.0 Request processed in 207us (356us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760713.598321:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 189 00000100:00100000:2.0:1789760713.598370:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4efb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519248128:0@lo:13:osp-pre-1-0.0 00000001:00080000:1.0:1789760713.633032:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972646 is committed 00002000:00010000:2.0:1789760713.634317:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21819392 -> 21823488 00002000:00010000:2.0:1789760713.634324:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760712 -> 1789760713 00002000:00010000:2.0:1789760713.634327:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760712 -> 1789760713 00010000:00010000:2.0:1789760713.634342:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.634356:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047d3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.634375:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.634384:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760713.634390:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.634400:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047d3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.634422:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047d3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.634432:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e047d3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760713.634589:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493662208:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42941us (43520us total) trans 4294972646 rc 0/0 00000100:00100000:2.0:1789760713.634601:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5357 00000100:00100000:2.0:1789760713.634611:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723893409475 time 723932670026 state 0x2 ref 0 00000100:00100000:2.0:1789760713.640481:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760713.640571:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493662336 00000100:00100000:0.0:1789760713.640581:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493662336 00000100:00100000:0.0:1789760713.640634:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723893409475 time 723932670026 state 0x0 ref 1 00000100:00100000:0.0:1789760713.640650:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723932930442 time 723932670026 state 0x1 ref 1 00000100:00100000:0.0:1789760713.640682:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723932930442 time 723981718241 state 0x0 ref 1 00000100:00100000:0.0:1789760713.640705:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493662336:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760713.640766:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21823488, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:0.0:1789760713.640891:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e047da lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.640909:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760713.640914:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760713.640920:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e047da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.640933:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e047da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760713.685011:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21823488 -> 21827584 00010000:00010000:2.0:1789760713.685033:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e047da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.685046:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e047da lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.685060:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e047da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.685067:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760713.685071:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e047da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.685080:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e047da lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.685103:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e047da lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.685111:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e047da lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760713.685209:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493662336:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44502us (44735us total) trans 4294972647 rc 0/0 00000100:00100000:2.0:1789760713.685219:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5358 00000100:00100000:2.0:1789760713.685229:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723932930442 time 723981718241 state 0x2 ref 0 00000001:00080000:1.0:1789760713.685388:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972647 is committed 00000100:00100000:1.0:1789760713.688706:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760713.688906:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493662464 00000100:00100000:0.0:1789760713.688916:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493662464 00000100:00100000:0.0:1789760713.688996:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723932930442 time 723981718241 state 0x0 ref 1 00000100:00100000:0.0:1789760713.689011:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723981978657 time 723981718241 state 0x1 ref 1 00000100:00100000:0.0:1789760713.689036:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723981978657 time 724030072077 state 0x0 ref 1 00000100:00100000:0.0:1789760713.689048:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc352f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493662464:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760713.689095:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21827584, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:0.0:1789760713.689104:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047e1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.689118:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760713.689122:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760713.689128:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.689141:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760713.801872:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21827584 -> 21831680 00010000:00010000:0.0:1789760713.801892:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.801905:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047e1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.801924:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.801932:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760713.801937:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.801946:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047e1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.801986:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047e1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.801993:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047e1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760713.802082:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493662464:12345-192.168.203.60@tcp:4:dd.0 Request processed in 113031us (113392us total) trans 4294972648 rc 0/0 00000100:00100000:0.0:1789760713.802091:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5359 00000100:00100000:0.0:1789760713.802100:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723981978657 time 724030072077 state 0x2 ref 0 00000001:00080000:2.0:1789760713.804748:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972648 is committed 00000100:00100000:0.0:1789760713.806507:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760713.806802:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493662592 00000100:00100000:3.0:1789760713.806812:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493662592 00000100:00100000:3.0:1789760713.806873:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 723981978657 time 724030072077 state 0x0 ref 1 00000100:00100000:3.0:1789760713.806889:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724030332493 time 724030072077 state 0x1 ref 1 00000100:00100000:3.0:1789760713.807113:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724030332493 time 724148146746 state 0x0 ref 1 00000100:00100000:3.0:1789760713.807125:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493662592:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760713.807308:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21831680, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:3.0:1789760713.807317:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e047e8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.807333:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760713.807337:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760713.807344:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e047e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.807358:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e047e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760713.878143:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972649 is committed 00002000:00010000:3.0:1789760713.878316:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21831680 -> 21835776 00010000:00010000:3.0:1789760713.878330:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e047e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.878342:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e047e8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.878351:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e047e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.878359:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760713.878363:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e047e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.878371:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e047e8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.878389:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e047e8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760713.878396:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e047e8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760713.878488:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493662592:12345-192.168.203.60@tcp:4:dd.0 Request processed in 71359us (71982us total) trans 4294972649 rc 0/0 00000100:00100000:3.0:1789760713.878502:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5360 00000100:00100000:3.0:1789760713.878512:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724030332493 time 724148146746 state 0x2 ref 0 00000100:00100000:0.0:1789760713.894464:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760713.894625:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493662720 00000100:00100000:2.0:1789760713.894636:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493662720 00000100:00100000:2.0:1789760713.894698:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724030332493 time 724148146746 state 0x0 ref 1 00000100:00100000:2.0:1789760713.894714:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724148407162 time 724148146746 state 0x1 ref 1 00000100:00100000:2.0:1789760713.894750:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724148407162 time 724235778328 state 0x0 ref 1 00000100:00100000:2.0:1789760713.894764:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493662720:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760713.894816:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21835776, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:2.0:1789760713.894825:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e047ef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.894840:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760713.894845:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760713.894851:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e047ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.894865:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e047ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760713.981045:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972650 is committed 00002000:00010000:1.0:1789760713.981561:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21835776 -> 21839872 00010000:00010000:1.0:1789760713.981577:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e047ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.981590:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e047ef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.981598:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e047ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.981605:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760713.981609:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e047ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.981617:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e047ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.981637:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e047ef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.981645:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e047ef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.981864:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493662720:12345-192.168.203.60@tcp:4:dd.0 Request processed in 87097us (87405us total) trans 4294972650 rc 0/0 00000100:00100000:1.0:1789760713.981876:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5361 00000100:00100000:1.0:1789760713.981885:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724148407162 time 724235778328 state 0x2 ref 0 00000100:00100000:2.0:1789760714.015521:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760714.015695:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493662848 00000100:00100000:0.0:1789760714.015705:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493662848 00000100:00100000:0.0:1789760714.015770:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724148407162 time 724235778328 state 0x0 ref 1 00000100:00100000:0.0:1789760714.015786:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724236038744 time 724235778328 state 0x1 ref 1 00000100:00100000:0.0:1789760714.015815:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724236038744 time 724356850443 state 0x0 ref 1 00000100:00100000:0.0:1789760714.015826:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493662848:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760714.015876:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21839872, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:0.0:1789760714.015885:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047f6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.015906:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760714.015922:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760714.015929:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.015942:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760714.082341:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972651 is committed 00002000:00010000:0.0:1789760714.082470:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21839872 -> 21843968 00010000:00010000:0.0:1789760714.082485:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.082498:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047f6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.082507:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.082514:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760714.082518:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.082527:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047f6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.082546:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047f6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.082554:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047f6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760714.082645:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493662848:12345-192.168.203.60@tcp:4:dd.0 Request processed in 66816us (67128us total) trans 4294972651 rc 0/0 00000100:00100000:0.0:1789760714.082655:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5362 00000100:00100000:0.0:1789760714.082665:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724236038744 time 724356850443 state 0x2 ref 0 00000100:00100000:2.0:1789760714.091368:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760714.091468:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493662976 00000100:00100000:0.0:1789760714.091477:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493662976 00000100:00100000:0.0:1789760714.091531:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724236038744 time 724356850443 state 0x0 ref 1 00000100:00100000:0.0:1789760714.091546:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724357110859 time 724356850443 state 0x1 ref 1 00000100:00100000:0.0:1789760714.091578:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724357110859 time 724432614041 state 0x0 ref 1 00000100:00100000:0.0:1789760714.091590:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493662976:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760714.091637:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21843968, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:0.0:1789760714.091645:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047fd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.091661:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760714.091665:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760714.091671:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.091700:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760714.157283:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972652 is committed 00002000:00010000:1.0:1789760714.157548:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21843968 -> 21848064 00010000:00010000:1.0:1789760714.157564:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.157577:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047fd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.157586:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.157592:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760714.157596:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.157610:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047fd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.157628:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047fd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.157636:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e047fd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.157745:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493662976:12345-192.168.203.60@tcp:4:dd.0 Request processed in 66144us (66374us total) trans 4294972652 rc 0/0 00000100:00100000:1.0:1789760714.157755:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5363 00000100:00100000:1.0:1789760714.157767:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724357110859 time 724432614041 state 0x2 ref 0 00000100:00100000:1.0:1789760714.163109:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760714.163264:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493663104 00000100:00100000:3.0:1789760714.163275:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493663104 00000100:00100000:3.0:1789760714.163327:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724357110859 time 724432614041 state 0x0 ref 1 00000100:00100000:3.0:1789760714.163341:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724432874457 time 724432614041 state 0x1 ref 1 00000100:00100000:3.0:1789760714.163368:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724432874457 time 724504404314 state 0x0 ref 1 00000100:00100000:3.0:1789760714.163379:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc352c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493663104:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760714.163426:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21848064, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:3.0:1789760714.163435:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04804 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.163449:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760714.163453:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.163458:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04804 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.163470:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04804 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760714.239296:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972653 is committed 00002000:00010000:3.0:1789760714.239375:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21848064 -> 21852160 00010000:00010000:3.0:1789760714.239390:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04804 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.239403:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04804 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.239412:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04804 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.239419:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760714.239424:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04804 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.239433:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04804 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.239453:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04804 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.239461:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04804 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.239552:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493663104:12345-192.168.203.60@tcp:4:dd.0 Request processed in 76169us (76448us total) trans 4294972653 rc 0/0 00000100:00100000:3.0:1789760714.239562:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5364 00000100:00100000:3.0:1789760714.239572:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724432874457 time 724504404314 state 0x2 ref 0 00000100:00100000:3.0:1789760714.245676:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760714.245895:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493663232 00000100:00100000:2.0:1789760714.245905:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493663232 00000100:00100000:2.0:1789760714.246013:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724432874457 time 724504404314 state 0x0 ref 1 00000100:00100000:2.0:1789760714.246030:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724504664730 time 724504404314 state 0x1 ref 1 00000100:00100000:2.0:1789760714.246054:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724504664730 time 724587089083 state 0x0 ref 1 00000100:00100000:2.0:1789760714.246067:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493663232:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760714.246114:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21852160, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:2.0:1789760714.246123:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0480b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.246138:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760714.246143:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.246149:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0480b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.246164:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0480b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760714.290589:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21852160 -> 21856256 00010000:00010000:1.0:1789760714.290617:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0480b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.290638:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0480b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.290646:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0480b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.290658:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760714.290662:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0480b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.290670:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0480b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.290698:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0480b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.290705:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0480b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.290810:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493663232:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44739us (45136us total) trans 4294972654 rc 0/0 00000100:00100000:1.0:1789760714.290821:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5365 00000100:00100000:1.0:1789760714.290831:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724504664730 time 724587089083 state 0x2 ref 0 00000001:00080000:3.0:1789760714.291815:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972654 is committed 00000100:00100000:3.0:1789760714.294522:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760714.294611:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493663360 00000100:00100000:1.0:1789760714.294621:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493663360 00000100:00100000:1.0:1789760714.294676:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724504664730 time 724587089083 state 0x0 ref 1 00000100:00100000:1.0:1789760714.294703:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724587349499 time 724587089083 state 0x1 ref 1 00000100:00100000:1.0:1789760714.294730:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724587349499 time 724635766066 state 0x0 ref 1 00000100:00100000:1.0:1789760714.294754:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493663360:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760714.294807:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21856256, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:1.0:1789760714.294816:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04812 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.294832:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760714.294836:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.294843:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04812 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.294855:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04812 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.302048:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760714.342055:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972655 is committed 00002000:00010000:1.0:1789760714.342365:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21856256 -> 21860352 00010000:00010000:1.0:1789760714.342381:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04812 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.342395:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04812 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.342404:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04812 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.342411:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760714.342416:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04812 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.342426:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04812 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.342444:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04812 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.342451:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04812 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.342538:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493663360:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47782us (48018us total) trans 4294972655 rc 0/0 00000100:00100000:1.0:1789760714.342548:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5366 00000100:00100000:1.0:1789760714.342558:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724587349499 time 724635766066 state 0x2 ref 0 00000100:00100000:2.0:1789760714.348815:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760714.348938:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493663488 00000100:00100000:0.0:1789760714.348949:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493663488 00000100:00100000:0.0:1789760714.349031:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724587349499 time 724635766066 state 0x0 ref 1 00000100:00100000:0.0:1789760714.349046:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724636026482 time 724635766066 state 0x1 ref 1 00000100:00100000:0.0:1789760714.349066:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724636026482 time 724690102313 state 0x0 ref 1 00000100:00100000:0.0:1789760714.349078:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9621018a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493663488:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760714.349124:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21860352, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:0.0:1789760714.349134:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04819 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.349149:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760714.349154:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760714.349160:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04819 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.349331:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04819 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760714.391863:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972656 is committed 00002000:00010000:0.0:1789760714.392078:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21860352 -> 21864448 00010000:00010000:0.0:1789760714.392094:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04819 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.392107:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04819 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.392116:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04819 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.392123:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760714.392128:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04819 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.392136:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04819 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.392155:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04819 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.392175:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04819 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760714.392266:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9621018a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493663488:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43185us (43456us total) trans 4294972656 rc 0/0 00000100:00100000:0.0:1789760714.392277:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5367 00000100:00100000:0.0:1789760714.392288:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724636026482 time 724690102313 state 0x2 ref 0 00000100:00100000:3.0:1789760714.395576:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760714.400052:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493663616 00000100:00100000:0.0:1789760714.400066:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493663616 00000100:00100000:0.0:1789760714.400130:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724636026482 time 724690102313 state 0x0 ref 1 00000100:00100000:0.0:1789760714.400149:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724690362729 time 724690102313 state 0x1 ref 1 00000100:00100000:0.0:1789760714.400296:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724690362729 time 724741329646 state 0x0 ref 1 00000100:00100000:0.0:1789760714.400312:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493663616:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760714.400361:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21864448, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:0.0:1789760714.400370:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04820 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.400386:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760714.400390:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760714.400396:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04820 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.400411:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04820 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760714.440020:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972657 is committed 00002000:00010000:2.0:1789760714.440133:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21864448 -> 21868544 00010000:00010000:2.0:1789760714.440148:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04820 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.440171:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04820 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.440181:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04820 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.440189:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760714.440194:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04820 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.440204:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04820 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.440223:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04820 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.440231:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04820 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.440320:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493663616:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40005us (44746us total) trans 4294972657 rc 0/0 00000100:00100000:2.0:1789760714.440330:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5368 00000100:00100000:2.0:1789760714.440349:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724690362729 time 724741329646 state 0x2 ref 0 00000100:00100000:2.0:1789760714.444293:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760714.444391:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493663744 00000100:00100000:1.0:1789760714.444402:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493663744 00000100:00100000:1.0:1789760714.444458:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724690362729 time 724741329646 state 0x0 ref 1 00000100:00100000:1.0:1789760714.444474:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724741590062 time 724741329646 state 0x1 ref 1 00000100:00100000:1.0:1789760714.444511:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724741590062 time 724785545927 state 0x0 ref 1 00000100:00100000:1.0:1789760714.444524:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9621018a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493663744:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760714.444578:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21868544, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:1.0:1789760714.444588:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04827 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.444605:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760714.444610:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.444616:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04827 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.444630:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04827 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760714.490436:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972658 is committed 00002000:00010000:1.0:1789760714.490528:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21868544 -> 21872640 00010000:00010000:1.0:1789760714.490544:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04827 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.490558:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04827 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.490568:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04827 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.490576:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760714.490581:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04827 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.490591:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04827 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.490616:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04827 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.490625:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04827 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.490764:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9621018a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493663744:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46238us (46478us total) trans 4294972658 rc 0/0 00000100:00100000:1.0:1789760714.491092:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5369 00000100:00100000:1.0:1789760714.491102:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724741590062 time 724785545927 state 0x2 ref 0 00000100:00100000:1.0:1789760714.495563:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760714.495644:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493663872 00000100:00100000:2.0:1789760714.495653:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493663872 00000100:00100000:2.0:1789760714.495715:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724741590062 time 724785545927 state 0x0 ref 1 00000100:00100000:2.0:1789760714.495829:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724785806343 time 724785545927 state 0x1 ref 1 00000100:00100000:2.0:1789760714.495905:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724785806343 time 724836940047 state 0x0 ref 1 00000100:00100000:2.0:1789760714.495918:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc352ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493663872:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760714.495992:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21872640, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:2.0:1789760714.496001:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0482e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.496016:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760714.496020:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.496026:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0482e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.496040:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0482e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760714.557611:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21872640 -> 21876736 00010000:00010000:2.0:1789760714.557630:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0482e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.557643:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0482e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.557653:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0482e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.557661:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760714.557666:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0482e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.557676:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0482e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.557711:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0482e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.557720:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0482e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.557835:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493663872:12345-192.168.203.60@tcp:4:dd.0 Request processed in 61914us (62276us total) trans 4294972659 rc 0/0 00000100:00100000:2.0:1789760714.557846:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5370 00000100:00100000:2.0:1789760714.557856:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724785806343 time 724836940047 state 0x2 ref 0 00000001:00080000:3.0:1789760714.560377:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972659 is committed 00000100:00100000:2.0:1789760714.561728:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760714.566333:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493664000 00000100:00100000:0.0:1789760714.566347:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493664000 00000100:00100000:0.0:1789760714.566408:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724785806343 time 724836940047 state 0x0 ref 1 00000100:00100000:0.0:1789760714.566422:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724837200463 time 724836940047 state 0x1 ref 1 00000100:00100000:0.0:1789760714.566447:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724837200463 time 724907482941 state 0x0 ref 1 00000100:00100000:0.0:1789760714.566459:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3662300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493664000:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760714.566510:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21876736, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:0.0:1789760714.566518:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04835 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.566534:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760714.566538:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760714.566544:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04835 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.566558:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04835 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760714.617122:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972660 is committed 00002000:00010000:1.0:1789760714.617280:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21876736 -> 21880832 00010000:00010000:1.0:1789760714.617294:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04835 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.617306:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04835 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.617315:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04835 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.617322:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760714.617326:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04835 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.617342:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04835 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.617361:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04835 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.617367:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04835 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.617456:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3662300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493664000:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50994us (55730us total) trans 4294972660 rc 0/0 00000100:00100000:1.0:1789760714.617466:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5371 00000100:00100000:1.0:1789760714.617474:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724837200463 time 724907482941 state 0x2 ref 0 00000100:00100000:0.0:1789760714.622230:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760714.622282:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493664128 00000100:00100000:2.0:1789760714.622293:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493664128 00000100:00100000:2.0:1789760714.622349:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724837200463 time 724907482941 state 0x0 ref 1 00000100:00100000:2.0:1789760714.622366:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724907743357 time 724907482941 state 0x1 ref 1 00000100:00100000:2.0:1789760714.622404:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724907743357 time 724963439557 state 0x0 ref 1 00000100:00100000:2.0:1789760714.622419:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493664128:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760714.622465:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21880832, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0xa807 00010000:00010000:2.0:1789760714.622476:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0483c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.622498:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760714.622503:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.622509:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0483c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.622524:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0483c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760714.663661:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972661 is committed 00002000:00010000:2.0:1789760714.663850:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21880832 -> 21884928 00002000:00010000:2.0:1789760714.663855:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760713 -> 1789760714 00002000:00010000:2.0:1789760714.663857:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760713 -> 1789760714 00010000:00010000:2.0:1789760714.663872:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0483c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.663886:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0483c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.663896:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0483c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.663905:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760714.663911:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0483c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.663921:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0483c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.663946:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0483c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.663954:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0483c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.664072:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493664128:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41652us (41844us total) trans 4294972661 rc 0/0 00000100:00100000:2.0:1789760714.664083:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5372 00000100:00100000:2.0:1789760714.664093:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724907743357 time 724963439557 state 0x2 ref 0 00000100:00100000:0.0:1789760714.668335:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760714.668429:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493664256 00000100:00100000:1.0:1789760714.668439:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493664256 00000100:00100000:1.0:1789760714.668498:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724907743357 time 724963439557 state 0x0 ref 1 00000100:00100000:1.0:1789760714.668515:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724963699973 time 724963439557 state 0x1 ref 1 00000100:00100000:1.0:1789760714.668552:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724963699973 time 725009587261 state 0x0 ref 1 00000100:00100000:1.0:1789760714.668566:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493664256:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760714.668621:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21884928, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:1.0:1789760714.668630:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04843 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.668646:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760714.668650:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.668656:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04843 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.668668:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04843 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760714.716491:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972662 is committed 00002000:00010000:3.0:1789760714.716651:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21884928 -> 21889024 00010000:00010000:3.0:1789760714.716668:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04843 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.716689:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04843 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.716698:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04843 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.716705:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760714.716711:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04843 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.716721:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04843 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.716741:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04843 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.716754:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04843 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.716848:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493664256:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48280us (48518us total) trans 4294972662 rc 0/0 00000100:00100000:3.0:1789760714.716859:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5373 00000100:00100000:3.0:1789760714.716868:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724963699973 time 725009587261 state 0x2 ref 0 00000100:00100000:3.0:1789760714.727333:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760714.727839:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493664384 00000100:00100000:2.0:1789760714.727850:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493664384 00000100:00100000:2.0:1789760714.727905:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 724963699973 time 725009587261 state 0x0 ref 1 00000100:00100000:2.0:1789760714.727920:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725009847677 time 725009587261 state 0x1 ref 1 00000100:00100000:2.0:1789760714.727957:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725009847677 time 725068992001 state 0x0 ref 1 00000100:00100000:2.0:1789760714.728007:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493664384:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760714.728058:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21889024, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:2.0:1789760714.728067:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0484a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.728081:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760714.728087:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.728093:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0484a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.728106:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0484a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760714.759055:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972663 is committed 00002000:00010000:0.0:1789760714.759245:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21889024 -> 21893120 00010000:00010000:0.0:1789760714.759260:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0484a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.759274:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0484a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.759283:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0484a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.759293:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760714.759299:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0484a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.759310:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0484a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.759329:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0484a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.759338:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0484a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760714.759466:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493664384:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31457us (32137us total) trans 4294972663 rc 0/0 00000100:00100000:0.0:1789760714.759478:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5374 00000100:00100000:0.0:1789760714.759487:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725009847677 time 725068992001 state 0x2 ref 0 00000100:00100000:0.0:1789760714.764697:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760714.764822:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493664512 00000100:00100000:1.0:1789760714.764831:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493664512 00000100:00100000:1.0:1789760714.764883:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725009847677 time 725068992001 state 0x0 ref 1 00000100:00100000:1.0:1789760714.764897:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725069252417 time 725068992001 state 0x1 ref 1 00000100:00100000:1.0:1789760714.764927:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725069252417 time 725105962607 state 0x0 ref 1 00000100:00100000:1.0:1789760714.764939:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a34e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493664512:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760714.765003:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21893120, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:1.0:1789760714.765012:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04851 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.765028:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760714.765032:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.765038:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04851 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.765053:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04851 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760714.804569:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972664 is committed 00002000:00010000:1.0:1789760714.804671:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21893120 -> 21897216 00010000:00010000:1.0:1789760714.804692:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04851 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.804706:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04851 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.804715:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04851 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.804723:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760714.804728:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04851 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.804745:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04851 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.804763:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04851 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.804879:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04851 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.805241:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a34e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493664512:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40299us (40559us total) trans 4294972664 rc 0/0 00000100:00100000:1.0:1789760714.805252:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5375 00000100:00100000:1.0:1789760714.805262:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725069252417 time 725105962607 state 0x2 ref 0 00000100:00100000:3.0:1789760714.808316:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760714.808452:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493664640 00000100:00100000:1.0:1789760714.808462:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493664640 00000100:00100000:1.0:1789760714.808515:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725069252417 time 725105962607 state 0x0 ref 1 00000100:00100000:1.0:1789760714.808530:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725106223023 time 725105962607 state 0x1 ref 1 00000100:00100000:1.0:1789760714.808564:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725106223023 time 725149599786 state 0x0 ref 1 00000100:00100000:1.0:1789760714.808577:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493664640:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760714.808626:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21897216, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:1.0:1789760714.808634:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04858 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.808649:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760714.808654:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.808659:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04858 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.808672:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04858 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760714.846665:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21897216 -> 21901312 00010000:00010000:1.0:1789760714.846701:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04858 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.846714:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04858 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.846722:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04858 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.846729:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760714.846734:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04858 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.846919:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04858 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.846940:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04858 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.846949:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04858 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.847077:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493664640:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38498us (38768us total) trans 4294972665 rc 0/0 00000100:00100000:1.0:1789760714.847089:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5376 00000100:00100000:1.0:1789760714.847099:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725106223023 time 725149599786 state 0x2 ref 0 00000001:00080000:0.0:1789760714.848557:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972665 is committed 00000100:00100000:3.0:1789760714.851131:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760714.851326:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493664768 00000100:00100000:1.0:1789760714.851336:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493664768 00000100:00100000:1.0:1789760714.851391:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725106223023 time 725149599786 state 0x0 ref 1 00000100:00100000:1.0:1789760714.851406:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725149860202 time 725149599786 state 0x1 ref 1 00000100:00100000:1.0:1789760714.851427:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725149860202 time 725192463145 state 0x0 ref 1 00000100:00100000:1.0:1789760714.851439:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493664768:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760714.851486:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21901312, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:1.0:1789760714.851495:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0485f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.851510:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760714.851514:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.851519:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0485f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.851531:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0485f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760714.886085:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972666 is committed 00002000:00010000:1.0:1789760714.886524:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21901312 -> 21905408 00010000:00010000:1.0:1789760714.886540:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0485f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.886554:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0485f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.886563:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0485f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.886571:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760714.886634:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0485f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.886642:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0485f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.886659:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0485f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.886664:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0485f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.886928:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493664768:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35483us (35799us total) trans 4294972666 rc 0/0 00000100:00100000:1.0:1789760714.886940:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5377 00000100:00100000:1.0:1789760714.886952:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725149860202 time 725192463145 state 0x2 ref 0 00000100:00100000:0.0:1789760714.891424:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760714.891664:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493664896 00000100:00100000:3.0:1789760714.891674:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493664896 00000100:00100000:3.0:1789760714.891792:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725149860202 time 725192463145 state 0x0 ref 1 00000100:00100000:3.0:1789760714.891808:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725192723561 time 725192463145 state 0x1 ref 1 00000100:00100000:3.0:1789760714.891839:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725192723561 time 725232874979 state 0x0 ref 1 00000100:00100000:3.0:1789760714.891852:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493664896:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760714.891904:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21905408, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:3.0:1789760714.891913:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04866 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.891934:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760714.891938:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760714.891945:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04866 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.891985:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04866 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760714.927408:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21905408 -> 21909504 00010000:00010000:3.0:1789760714.927428:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04866 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.927441:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04866 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.927450:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04866 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.927458:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760714.927463:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04866 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.927472:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04866 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.927491:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04866 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760714.927499:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04866 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760714.927594:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493664896:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35738us (36172us total) trans 4294972667 rc 0/0 00000100:00100000:3.0:1789760714.927606:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5378 00000100:00100000:3.0:1789760714.927615:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725192723561 time 725232874979 state 0x2 ref 0 00000001:00080000:2.0:1789760714.927707:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972667 is committed 00000100:00100000:0.0:1789760714.932001:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760714.932184:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493665024 00000100:00100000:1.0:1789760714.932194:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493665024 00000100:00100000:1.0:1789760714.932252:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725192723561 time 725232874979 state 0x0 ref 1 00000100:00100000:1.0:1789760714.932268:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725233135395 time 725232874979 state 0x1 ref 1 00000100:00100000:1.0:1789760714.932384:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725233135395 time 725273417941 state 0x0 ref 1 00000100:00100000:1.0:1789760714.932397:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493665024:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760714.932443:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21909504, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:1.0:1789760714.932452:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0486d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.932467:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760714.932471:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.932477:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0486d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.932492:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0486d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760714.974037:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21909504 -> 21913600 00010000:00010000:0.0:1789760714.974057:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0486d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.974071:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0486d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.974080:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0486d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.974088:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760714.974093:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0486d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.974101:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0486d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.974134:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0486d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.974142:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0486d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760714.974234:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493665024:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41834us (42334us total) trans 4294972668 rc 0/0 00000100:00100000:0.0:1789760714.974245:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5379 00000100:00100000:0.0:1789760714.974255:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725233135395 time 725273417941 state 0x2 ref 0 00000001:00080000:2.0:1789760714.974607:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972668 is committed 00000100:00100000:1.0:1789760714.980558:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760714.980649:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493665152 00000100:00100000:0.0:1789760714.980660:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493665152 00000100:00100000:0.0:1789760714.980730:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725233135395 time 725273417941 state 0x0 ref 1 00000100:00100000:0.0:1789760714.980806:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725273678357 time 725273417941 state 0x1 ref 1 00000100:00100000:0.0:1789760714.980876:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725273678357 time 725321910954 state 0x0 ref 1 00000100:00100000:0.0:1789760714.980890:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493665152:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760714.980939:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21913600, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:0.0:1789760714.980948:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04874 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.981003:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760714.981008:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760714.981014:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04874 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.981029:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04874 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760715.043171:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972669 is committed 00002000:00010000:0.0:1789760715.043341:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21913600 -> 21917696 00010000:00010000:0.0:1789760715.043356:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04874 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.043368:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04874 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.043382:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04874 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.043389:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760715.043394:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04874 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.043402:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04874 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.043421:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04874 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.043428:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04874 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760715.043517:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493665152:12345-192.168.203.60@tcp:4:dd.0 Request processed in 62626us (62963us total) trans 4294972669 rc 0/0 00000100:00100000:0.0:1789760715.043527:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5380 00000100:00100000:0.0:1789760715.043537:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725273678357 time 725321910954 state 0x2 ref 0 00000100:00100000:0.0:1789760715.047361:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760715.047419:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493665280 00000100:00100000:2.0:1789760715.047428:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493665280 00000100:00100000:2.0:1789760715.047541:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725273678357 time 725321910954 state 0x0 ref 1 00000100:00100000:2.0:1789760715.047552:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725322171370 time 725321910954 state 0x1 ref 1 00000100:00100000:2.0:1789760715.047574:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725322171370 time 725388610593 state 0x0 ref 1 00000100:00100000:2.0:1789760715.047584:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493665280:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760715.047624:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21917696, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:2.0:1789760715.047631:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0487b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.047643:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760715.047646:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.047650:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0487b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.047661:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0487b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760715.048460:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760715.098417:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972670 is committed 00002000:00010000:2.0:1789760715.098545:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21917696 -> 21921792 00010000:00010000:2.0:1789760715.098561:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0487b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.098646:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0487b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.098656:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0487b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.098663:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760715.098668:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0487b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.098687:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0487b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.098707:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0487b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.098714:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0487b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.099013:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493665280:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51424us (51655us total) trans 4294972670 rc 0/0 00000100:00100000:2.0:1789760715.099027:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5381 00000100:00100000:2.0:1789760715.099038:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725322171370 time 725388610593 state 0x2 ref 0 00000100:00100000:3.0:1789760715.109098:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760715.109417:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493665408 00000100:00100000:0.0:1789760715.109427:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493665408 00000100:00100000:0.0:1789760715.109481:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725322171370 time 725388610593 state 0x0 ref 1 00000100:00100000:0.0:1789760715.109497:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725388871009 time 725388610593 state 0x1 ref 1 00000100:00100000:0.0:1789760715.109582:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725388871009 time 725450616843 state 0x0 ref 1 00000100:00100000:0.0:1789760715.109595:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493665408:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760715.109644:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21921792, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:0.0:1789760715.109652:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04882 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.109668:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760715.109672:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760715.109688:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04882 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.109702:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04882 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760715.164623:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21921792 -> 21925888 00010000:00010000:0.0:1789760715.164642:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04882 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.164656:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04882 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.164665:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04882 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.164683:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760715.164688:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04882 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.164697:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04882 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.164715:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04882 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.164723:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04882 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760715.165335:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972671 is committed 00000100:00100000:0.0:1789760715.165491:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493665408:12345-192.168.203.60@tcp:4:dd.0 Request processed in 55894us (56397us total) trans 4294972671 rc 0/0 00000100:00100000:0.0:1789760715.165503:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5382 00000100:00100000:0.0:1789760715.165512:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725388871009 time 725450616843 state 0x2 ref 0 00000100:00100000:0.0:1789760715.169083:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760715.169292:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493665536 00000100:00100000:1.0:1789760715.169301:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493665536 00000100:00100000:1.0:1789760715.169354:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725388871009 time 725450616843 state 0x0 ref 1 00000100:00100000:1.0:1789760715.169369:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725450877259 time 725450616843 state 0x1 ref 1 00000100:00100000:1.0:1789760715.169452:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725450877259 time 725510486508 state 0x0 ref 1 00000100:00100000:1.0:1789760715.169465:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493665536:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760715.169511:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21925888, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:1.0:1789760715.169520:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04889 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.169536:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760715.169540:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760715.169546:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04889 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.169560:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04889 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760715.219556:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21925888 -> 21929984 00010000:00010000:0.0:1789760715.219585:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04889 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.219599:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04889 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.219615:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04889 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.219624:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760715.219629:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04889 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.219639:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04889 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.219658:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04889 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.219666:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04889 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760715.219775:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493665536:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50307us (50697us total) trans 4294972672 rc 0/0 00000100:00100000:0.0:1789760715.219786:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5383 00000100:00100000:0.0:1789760715.219796:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725450877259 time 725510486508 state 0x2 ref 0 00000001:00080000:2.0:1789760715.220693:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972672 is committed 00000100:00100000:0.0:1789760715.223353:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760715.223444:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493665664 00000100:00100000:1.0:1789760715.223454:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493665664 00000100:00100000:1.0:1789760715.223506:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725450877259 time 725510486508 state 0x0 ref 1 00000100:00100000:1.0:1789760715.223519:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725510746924 time 725510486508 state 0x1 ref 1 00000100:00100000:1.0:1789760715.224407:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725510746924 time 725565438473 state 0x0 ref 1 00000100:00100000:1.0:1789760715.224427:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb1b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493665664:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760715.224478:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21929984, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:1.0:1789760715.224487:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04890 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.224503:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760715.224508:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760715.224514:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04890 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.224527:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04890 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760715.284458:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972673 is committed 00002000:00010000:1.0:1789760715.284784:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21929984 -> 21934080 00010000:00010000:1.0:1789760715.284799:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04890 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.284812:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04890 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.284821:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04890 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.284828:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760715.284834:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04890 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.284844:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04890 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.284869:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04890 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.284877:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04890 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760715.285050:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb1b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493665664:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60625us (61702us total) trans 4294972673 rc 0/0 00000100:00100000:1.0:1789760715.285060:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5384 00000100:00100000:1.0:1789760715.285070:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725510746924 time 725565438473 state 0x2 ref 0 00000100:00100000:1.0:1789760715.292299:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760715.292412:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493665792 00000100:00100000:2.0:1789760715.292423:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493665792 00000100:00100000:2.0:1789760715.292480:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725510746924 time 725565438473 state 0x0 ref 1 00000100:00100000:2.0:1789760715.292495:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725565698889 time 725565438473 state 0x1 ref 1 00000100:00100000:2.0:1789760715.292562:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725565698889 time 725633596940 state 0x0 ref 1 00000100:00100000:2.0:1789760715.292584:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493665792:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760715.292632:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21934080, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:2.0:1789760715.292641:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04897 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.292657:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760715.292662:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.292668:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04897 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.292690:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04897 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.322247:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760715.327760:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972674 is committed 00002000:00010000:0.0:1789760715.328088:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21934080 -> 21938176 00010000:00010000:0.0:1789760715.328119:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04897 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.328207:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04897 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.328217:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04897 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.328224:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760715.328229:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04897 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.328240:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04897 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.328261:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04897 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.328270:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04897 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760715.328363:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493665792:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35785us (36069us total) trans 4294972674 rc 0/0 00000100:00100000:0.0:1789760715.328373:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5385 00000100:00100000:0.0:1789760715.328382:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725565698889 time 725633596940 state 0x2 ref 0 00000100:00100000:0.0:1789760715.333127:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760715.333387:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493665920 00000100:00100000:3.0:1789760715.333397:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493665920 00000100:00100000:3.0:1789760715.333449:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725565698889 time 725633596940 state 0x0 ref 1 00000100:00100000:3.0:1789760715.333464:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725633857356 time 725633596940 state 0x1 ref 1 00000100:00100000:3.0:1789760715.333536:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725633857356 time 725674570068 state 0x0 ref 1 00000100:00100000:3.0:1789760715.333550:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493665920:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760715.333595:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21938176, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:3.0:1789760715.333605:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0489e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.333620:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760715.333626:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760715.333633:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0489e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760715.333646:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0489e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760715.397629:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972675 is committed 00002000:00010000:0.0:1789760715.397993:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21938176 -> 21942272 00010000:00010000:0.0:1789760715.398010:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0489e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.398025:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0489e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.398035:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0489e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.398043:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760715.398048:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0489e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.398058:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0489e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.398077:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0489e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.398084:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0489e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760715.398318:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493665920:12345-192.168.203.60@tcp:4:dd.0 Request processed in 64765us (65213us total) trans 4294972675 rc 0/0 00000100:00100000:0.0:1789760715.398329:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5386 00000100:00100000:0.0:1789760715.398339:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725633857356 time 725674570068 state 0x2 ref 0 00000100:00100000:2.0:1789760715.409436:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760715.409511:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493666048 00000100:00100000:0.0:1789760715.409521:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493666048 00000100:00100000:0.0:1789760715.409574:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725633857356 time 725674570068 state 0x0 ref 1 00000100:00100000:0.0:1789760715.409589:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725674830484 time 725674570068 state 0x1 ref 1 00000100:00100000:0.0:1789760715.409620:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725674830484 time 725750655853 state 0x0 ref 1 00000100:00100000:0.0:1789760715.409633:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3661500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493666048:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760715.409690:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21942272, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:0.0:1789760715.409699:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e048a5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.409714:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760715.409718:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760715.409724:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e048a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.409737:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e048a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760715.496917:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972676 is committed 00002000:00010000:0.0:1789760715.498059:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21942272 -> 21946368 00010000:00010000:0.0:1789760715.498077:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e048a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.498091:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e048a5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.498645:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e048a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.498658:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760715.498664:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e048a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.498683:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e048a5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.498706:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e048a5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.499041:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e048a5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760715.500742:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3661500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493666048:12345-192.168.203.60@tcp:4:dd.0 Request processed in 90075us (90279us total) trans 4294972676 rc 0/0 00000100:00100000:0.0:1789760715.501372:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5387 00000100:00100000:0.0:1789760715.501390:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725674830484 time 725750655853 state 0x2 ref 0 00000100:00100000:3.0:1789760715.505035:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760715.505303:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493666176 00000100:00100000:1.0:1789760715.505313:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493666176 00000100:00100000:1.0:1789760715.505369:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725674830484 time 725750655853 state 0x0 ref 1 00000100:00100000:1.0:1789760715.505383:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725750916269 time 725750655853 state 0x1 ref 1 00000100:00100000:1.0:1789760715.505413:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725750916269 time 725846448583 state 0x0 ref 1 00000100:00100000:1.0:1789760715.505425:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493666176:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760715.505472:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21946368, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:1.0:1789760715.505483:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e048ac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.505499:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760715.505503:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760715.505508:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e048ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.505522:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e048ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760715.561653:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972677 is committed 00002000:00010000:1.0:1789760715.561899:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21946368 -> 21950464 00010000:00010000:1.0:1789760715.561914:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e048ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.561927:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e048ac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.561936:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e048ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.561944:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760715.561949:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e048ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.561993:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e048ac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.562014:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e048ac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.562024:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e048ac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760715.562122:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493666176:12345-192.168.203.60@tcp:4:dd.0 Request processed in 56693us (57091us total) trans 4294972677 rc 0/0 00000100:00100000:1.0:1789760715.562139:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5388 00000100:00100000:1.0:1789760715.562149:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725750916269 time 725846448583 state 0x2 ref 0 00000100:00100000:1.0:1789760715.566304:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760715.566408:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493666304 00000100:00100000:0.0:1789760715.566417:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493666304 00000100:00100000:0.0:1789760715.566474:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725750916269 time 725846448583 state 0x0 ref 1 00000100:00100000:0.0:1789760715.566495:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725846708999 time 725846448583 state 0x1 ref 1 00000100:00100000:0.0:1789760715.566587:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725846708999 time 725907622671 state 0x0 ref 1 00000100:00100000:0.0:1789760715.566600:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493666304:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760715.566647:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21950464, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:0.0:1789760715.566656:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e048b3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.566680:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760715.566684:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760715.566691:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e048b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.566705:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e048b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760715.605485:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21950464 -> 21954560 00010000:00010000:1.0:1789760715.605506:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e048b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.605520:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e048b3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.605529:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e048b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.605536:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760715.605541:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e048b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.605549:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e048b3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.605586:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e048b3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.605594:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e048b3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760715.605693:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493666304:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39089us (39394us total) trans 4294972678 rc 0/0 00000100:00100000:1.0:1789760715.605703:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5389 00000100:00100000:1.0:1789760715.606349:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725846708999 time 725907622671 state 0x2 ref 0 00000001:00080000:0.0:1789760715.608830:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972678 is committed 00000100:00100000:0.0:1789760715.609439:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760715.609560:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493666432 00000100:00100000:2.0:1789760715.609635:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493666432 00000100:00100000:2.0:1789760715.609702:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725846708999 time 725907622671 state 0x0 ref 1 00000100:00100000:2.0:1789760715.609783:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725907883087 time 725907622671 state 0x1 ref 1 00000100:00100000:2.0:1789760715.609844:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725907883087 time 725950878737 state 0x0 ref 1 00000100:00100000:2.0:1789760715.609858:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493666432:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760715.610035:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21954560, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0xa807 00010000:00010000:2.0:1789760715.610045:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e048ba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.610060:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760715.610065:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.610072:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e048ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.610085:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e048ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760715.688602:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972679 is committed 00002000:00010000:1.0:1789760715.690549:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21954560 -> 21958656 00002000:00010000:1.0:1789760715.690557:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760714 -> 1789760715 00002000:00010000:1.0:1789760715.690560:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760714 -> 1789760715 00010000:00010000:1.0:1789760715.690574:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e048ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.690590:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e048ba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.690600:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e048ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.690608:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760715.690613:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e048ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.690624:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e048ba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.690650:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e048ba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.690657:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e048ba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760715.690767:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493666432:12345-192.168.203.60@tcp:4:dd.0 Request processed in 80906us (81328us total) trans 4294972679 rc 0/0 00000100:00100000:1.0:1789760715.690777:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5390 00000100:00100000:1.0:1789760715.690787:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725907883087 time 725950878737 state 0x2 ref 0 00000100:00100000:3.0:1789760715.698108:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760715.698255:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493666560 00000100:00100000:2.0:1789760715.698266:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493666560 00000100:00100000:2.0:1789760715.698322:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725907883087 time 725950878737 state 0x0 ref 1 00000100:00100000:2.0:1789760715.698337:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725951139153 time 725950878737 state 0x1 ref 1 00000100:00100000:2.0:1789760715.698372:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725951139153 time 726039407716 state 0x0 ref 1 00000100:00100000:2.0:1789760715.698386:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493666560:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760715.698433:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21958656, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:2.0:1789760715.698442:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e048c1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.698458:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760715.698462:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.698469:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e048c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.698484:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e048c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760715.757480:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972680 is committed 00002000:00010000:0.0:1789760715.757583:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21958656 -> 21962752 00010000:00010000:0.0:1789760715.757597:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e048c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.757611:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e048c1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.757628:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e048c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.757635:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760715.757640:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e048c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.757651:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e048c1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.757682:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e048c1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.757691:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e048c1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760715.758012:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493666560:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59623us (59918us total) trans 4294972680 rc 0/0 00000100:00100000:0.0:1789760715.758022:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5391 00000100:00100000:0.0:1789760715.758032:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725951139153 time 726039407716 state 0x2 ref 0 00000100:00100000:0.0:1789760715.763664:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760715.764317:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493666688 00000100:00100000:1.0:1789760715.764327:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493666688 00000100:00100000:1.0:1789760715.764384:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 725951139153 time 726039407716 state 0x0 ref 1 00000100:00100000:1.0:1789760715.764398:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726039668132 time 726039407716 state 0x1 ref 1 00000100:00100000:1.0:1789760715.764435:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726039668132 time 726105469786 state 0x0 ref 1 00000100:00100000:1.0:1789760715.764448:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493666688:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760715.764501:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21962752, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:1.0:1789760715.764511:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048c8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.764528:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760715.764532:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760715.764539:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.764553:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760715.824700:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972681 is committed 00002000:00010000:1.0:1789760715.825432:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21962752 -> 21966848 00010000:00010000:1.0:1789760715.825447:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.825461:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.825470:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.825477:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760715.825482:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.825492:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.825518:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048c8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.825525:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048c8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760715.825619:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493666688:12345-192.168.203.60@tcp:4:dd.0 Request processed in 61168us (61958us total) trans 4294972681 rc 0/0 00000100:00100000:1.0:1789760715.825630:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5392 00000100:00100000:1.0:1789760715.825639:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726039668132 time 726105469786 state 0x2 ref 0 00000100:00100000:1.0:1789760715.831635:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760715.832436:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493666816 00000100:00100000:2.0:1789760715.832447:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493666816 00000100:00100000:2.0:1789760715.832507:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726039668132 time 726105469786 state 0x0 ref 1 00000100:00100000:2.0:1789760715.832523:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726105730202 time 726105469786 state 0x1 ref 1 00000100:00100000:2.0:1789760715.832594:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726105730202 time 726173629229 state 0x0 ref 1 00000100:00100000:2.0:1789760715.832608:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493666816:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760715.832661:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21966848, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:2.0:1789760715.832681:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e048cf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.832698:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760715.832703:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.832716:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e048cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.832737:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e048cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760715.923011:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21966848 -> 21970944 00010000:00010000:1.0:1789760715.923030:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e048cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.923043:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e048cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.923053:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e048cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.923060:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760715.923065:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e048cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.923080:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e048cf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.923110:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e048cf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.923118:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e048cf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760715.923205:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493666816:12345-192.168.203.60@tcp:4:dd.0 Request processed in 90594us (91574us total) trans 4294972682 rc 0/0 00000100:00100000:1.0:1789760715.923215:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5393 00000100:00100000:1.0:1789760715.923225:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726105730202 time 726173629229 state 0x2 ref 0 00000001:00080000:0.0:1789760715.923314:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972682 is committed 00000100:00100000:0.0:1789760715.930632:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760715.930752:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493666944 00000100:00100000:2.0:1789760715.930762:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493666944 00000100:00100000:2.0:1789760715.930818:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726105730202 time 726173629229 state 0x0 ref 1 00000100:00100000:2.0:1789760715.930834:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726173889645 time 726173629229 state 0x1 ref 1 00000100:00100000:2.0:1789760715.930864:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726173889645 time 726271899593 state 0x0 ref 1 00000100:00100000:2.0:1789760715.930875:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493666944:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760715.930929:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21970944, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:2.0:1789760715.930939:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e048d6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.930954:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760715.930991:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.930998:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e048d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.931012:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e048d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760715.983820:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972683 is committed 00002000:00010000:1.0:1789760715.984087:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21970944 -> 21975040 00010000:00010000:1.0:1789760715.984103:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e048d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.984116:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e048d6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.984126:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e048d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.984133:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760715.984139:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e048d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.984158:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e048d6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.984178:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e048d6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.984185:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e048d6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760715.984276:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493666944:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53397us (53649us total) trans 4294972683 rc 0/0 00000100:00100000:1.0:1789760715.984287:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5394 00000100:00100000:1.0:1789760715.984297:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726173889645 time 726271899593 state 0x2 ref 0 00000100:00100000:2.0:1789760715.988361:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760715.988467:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493667072 00000100:00100000:0.0:1789760715.988479:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493667072 00000100:00100000:0.0:1789760715.988546:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726173889645 time 726271899593 state 0x0 ref 1 00000100:00100000:0.0:1789760715.988568:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726272160009 time 726271899593 state 0x1 ref 1 00000100:00100000:0.0:1789760715.988601:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726272160009 time 726329633069 state 0x0 ref 1 00000100:00100000:0.0:1789760715.988616:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3662680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493667072:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760715.988685:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21975040, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:0.0:1789760715.988699:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e048dd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.988729:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760715.988735:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760715.988745:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e048dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.988761:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e048dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760716.057485:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972684 is committed 00002000:00010000:1.0:1789760716.057719:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21975040 -> 21979136 00010000:00010000:1.0:1789760716.057742:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e048dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.057754:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e048dd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.057770:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e048dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.057781:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760716.057786:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e048dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.057795:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e048dd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.057816:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e048dd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.057824:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e048dd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.057920:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3662680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493667072:12345-192.168.203.60@tcp:4:dd.0 Request processed in 69302us (69566us total) trans 4294972684 rc 0/0 00000100:00100000:1.0:1789760716.057930:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5395 00000100:00100000:1.0:1789760716.057940:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726272160009 time 726329633069 state 0x2 ref 0 00000100:00100000:0.0:1789760716.064622:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760716.064698:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493667200 00000100:00100000:1.0:1789760716.064708:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493667200 00000100:00100000:1.0:1789760716.064781:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726272160009 time 726329633069 state 0x0 ref 1 00000100:00100000:1.0:1789760716.064797:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726329893485 time 726329633069 state 0x1 ref 1 00000100:00100000:1.0:1789760716.064828:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726329893485 time 726405863208 state 0x0 ref 1 00000100:00100000:1.0:1789760716.064845:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493667200:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760716.064909:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21979136, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:1.0:1789760716.064920:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048e4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.064935:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760716.064939:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.064945:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.064991:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760716.192596:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972685 is committed 00002000:00010000:1.0:1789760716.192706:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21979136 -> 21983232 00010000:00010000:1.0:1789760716.192727:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.192739:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048e4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.192747:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.192762:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760716.192768:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.192778:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048e4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.192798:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048e4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.192806:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048e4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.192903:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493667200:12345-192.168.203.60@tcp:4:dd.0 Request processed in 128055us (128285us total) trans 4294972685 rc 0/0 00000100:00100000:1.0:1789760716.192913:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5396 00000100:00100000:1.0:1789760716.192921:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726329893485 time 726405863208 state 0x2 ref 0 00000100:00100000:0.0:1789760716.199363:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760716.199712:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493667328 00000100:00100000:1.0:1789760716.199729:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493667328 00000100:00100000:1.0:1789760716.199797:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726329893485 time 726405863208 state 0x0 ref 1 00000100:00100000:1.0:1789760716.199813:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726406123624 time 726405863208 state 0x1 ref 1 00000100:00100000:1.0:1789760716.199844:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726406123624 time 726540878884 state 0x0 ref 1 00000100:00100000:1.0:1789760716.199857:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493667328:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760716.199916:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21983232, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:1.0:1789760716.199925:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048eb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.199940:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760716.199944:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.199950:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.199994:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760716.262600:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972686 is committed 00002000:00010000:1.0:1789760716.262728:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21983232 -> 21987328 00010000:00010000:1.0:1789760716.262753:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.262767:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048eb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.262781:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.262788:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760716.262792:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.262800:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048eb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.262818:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048eb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.262826:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048eb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.262929:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493667328:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63069us (63571us total) trans 4294972686 rc 0/0 00000100:00100000:1.0:1789760716.262940:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5397 00000100:00100000:1.0:1789760716.262949:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726406123624 time 726540878884 state 0x2 ref 0 00000100:00100000:0.0:1789760716.269302:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760716.269403:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493667456 00000100:00100000:2.0:1789760716.269414:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493667456 00000100:00100000:2.0:1789760716.269468:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726406123624 time 726540878884 state 0x0 ref 1 00000100:00100000:2.0:1789760716.269482:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726541139300 time 726540878884 state 0x1 ref 1 00000100:00100000:2.0:1789760716.269551:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726541139300 time 726610586111 state 0x0 ref 1 00000100:00100000:2.0:1789760716.269565:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493667456:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760716.269617:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21987328, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:2.0:1789760716.269626:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e048f2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.269642:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760716.269646:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760716.269652:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e048f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.269677:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e048f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760716.341345:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972687 is committed 00002000:00010000:1.0:1789760716.341454:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21987328 -> 21991424 00010000:00010000:1.0:1789760716.341469:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e048f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.341482:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e048f2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.341492:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e048f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.341498:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760716.341503:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e048f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.341512:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e048f2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.341536:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e048f2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.341544:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e048f2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.341635:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493667456:12345-192.168.203.60@tcp:4:dd.0 Request processed in 72069us (72340us total) trans 4294972687 rc 0/0 00000100:00100000:1.0:1789760716.341646:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5398 00000100:00100000:1.0:1789760716.341655:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726541139300 time 726610586111 state 0x2 ref 0 00000100:00100000:0.0:1789760716.345214:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760716.345291:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493667584 00000100:00100000:1.0:1789760716.345302:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493667584 00000100:00100000:1.0:1789760716.345360:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726541139300 time 726610586111 state 0x0 ref 1 00000100:00100000:1.0:1789760716.345375:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726610846527 time 726610586111 state 0x1 ref 1 00000100:00100000:1.0:1789760716.345410:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726610846527 time 726686445748 state 0x0 ref 1 00000100:00100000:1.0:1789760716.345423:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493667584:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760716.345474:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21991424, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:1.0:1789760716.345484:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048f9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.345500:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760716.345505:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.345512:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.345526:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.359268:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760716.422356:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972688 is committed 00002000:00010000:1.0:1789760716.425024:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21991424 -> 21995520 00010000:00010000:1.0:1789760716.425044:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.425057:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048f9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.425067:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.425075:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760716.425080:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.425090:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048f9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.425112:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048f9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.425126:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e048f9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.425236:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493667584:12345-192.168.203.60@tcp:4:dd.0 Request processed in 79809us (80025us total) trans 4294972688 rc 0/0 00000100:00100000:1.0:1789760716.425248:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5399 00000100:00100000:1.0:1789760716.425258:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726610846527 time 726686445748 state 0x2 ref 0 00000100:00100000:2.0:1789760716.428682:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760716.428859:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493667712 00000100:00100000:1.0:1789760716.428870:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493667712 00000100:00100000:1.0:1789760716.429071:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726610846527 time 726686445748 state 0x0 ref 1 00000100:00100000:1.0:1789760716.429087:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726686706164 time 726686445748 state 0x1 ref 1 00000100:00100000:1.0:1789760716.429163:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726686706164 time 726770197891 state 0x0 ref 1 00000100:00100000:1.0:1789760716.429178:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3662300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493667712:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760716.429229:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21995520, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:1.0:1789760716.429237:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04900 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.429255:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760716.429260:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.429269:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04900 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.429295:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04900 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760716.487679:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972689 is committed 00002000:00010000:1.0:1789760716.487916:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21995520 -> 21999616 00010000:00010000:1.0:1789760716.487932:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04900 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.487944:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04900 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.487952:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04900 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.487991:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760716.487997:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04900 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.488006:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04900 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.488026:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04900 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.488034:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04900 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.488133:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3662300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493667712:12345-192.168.203.60@tcp:4:dd.0 Request processed in 58953us (59466us total) trans 4294972689 rc 0/0 00000100:00100000:1.0:1789760716.490829:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5400 00000100:00100000:1.0:1789760716.490845:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726686706164 time 726770197891 state 0x2 ref 0 00000100:00100000:0.0:1789760716.491475:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760716.491591:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493667840 00000100:00100000:2.0:1789760716.491600:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493667840 00000100:00100000:2.0:1789760716.491648:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726686706164 time 726770197891 state 0x0 ref 1 00000100:00100000:2.0:1789760716.491662:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726770458307 time 726770197891 state 0x1 ref 1 00000100:00100000:2.0:1789760716.491700:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726770458307 time 726832735124 state 0x0 ref 1 00000100:00100000:2.0:1789760716.491857:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493667840:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760716.491908:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 21999616, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:2.0:1789760716.491917:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04907 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.491932:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760716.491936:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760716.491942:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04907 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.491956:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04907 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760716.534877:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972690 is committed 00002000:00010000:3.0:1789760716.535014:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 21999616 -> 22003712 00010000:00010000:3.0:1789760716.535030:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04907 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.535043:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04907 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.535052:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04907 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.535059:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760716.535064:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04907 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.535073:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04907 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.535097:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04907 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.535105:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04907 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760716.536528:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493667840:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44669us (45050us total) trans 4294972690 rc 0/0 00000100:00100000:3.0:1789760716.536539:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5401 00000100:00100000:3.0:1789760716.536548:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726770458307 time 726832735124 state 0x2 ref 0 00000100:00100000:0.0:1789760716.541397:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760716.541613:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493667968 00000100:00100000:2.0:1789760716.541625:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493667968 00000100:00100000:2.0:1789760716.541689:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726770458307 time 726832735124 state 0x0 ref 1 00000100:00100000:2.0:1789760716.541716:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726832995540 time 726832735124 state 0x1 ref 1 00000100:00100000:2.0:1789760716.541748:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726832995540 time 726882783407 state 0x0 ref 1 00000100:00100000:2.0:1789760716.541761:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493667968:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760716.541811:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22003712, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:2.0:1789760716.541820:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0490e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.541836:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760716.541841:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760716.541848:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0490e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.541861:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0490e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760716.613024:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22003712 -> 22007808 00010000:00010000:3.0:1789760716.613046:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0490e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.613059:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0490e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.613069:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0490e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.613077:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760716.613082:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0490e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.613096:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0490e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.613115:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0490e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.613123:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0490e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760716.613223:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493667968:12345-192.168.203.60@tcp:4:dd.0 Request processed in 71460us (71831us total) trans 4294972691 rc 0/0 00000100:00100000:3.0:1789760716.613234:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5402 00000100:00100000:3.0:1789760716.613245:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726832995540 time 726882783407 state 0x2 ref 0 00000001:00080000:0.0:1789760716.616418:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972691 is committed 00000100:00100000:2.0:1789760716.616930:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760716.617123:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493668096 00000100:00100000:1.0:1789760716.617134:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493668096 00000100:00100000:2.0:1789760716.617302:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726832995540 time 726882783407 state 0x0 ref 1 00000100:00100000:2.0:1789760716.617318:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726883043823 time 726882783407 state 0x1 ref 1 00000100:00100000:2.0:1789760716.617379:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726883043823 time 726958414418 state 0x0 ref 1 00000100:00100000:2.0:1789760716.617393:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c89f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493668096:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760716.617442:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22007808, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0xa807 00010000:00010000:2.0:1789760716.617450:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04915 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.617465:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760716.617469:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760716.617475:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04915 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.617488:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04915 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760716.680480:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972692 is committed 00002000:00010000:2.0:1789760716.680578:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22007808 -> 22011904 00002000:00010000:2.0:1789760716.680583:0:13038:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760715 -> 1789760716 00002000:00010000:2.0:1789760716.680586:0:13038:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760715 -> 1789760716 00010000:00010000:2.0:1789760716.680599:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04915 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.680611:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04915 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.680620:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04915 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.680628:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760716.680633:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04915 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.680642:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04915 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.680678:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04915 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.680686:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04915 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760716.680843:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c89f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493668096:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63448us (63914us total) trans 4294972692 rc 0/0 00000100:00100000:2.0:1789760716.680854:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5403 00000100:00100000:2.0:1789760716.680862:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726883043823 time 726958414418 state 0x2 ref 0 00000100:00100000:1.0:1789760716.685789:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760716.686569:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493668224 00000100:00100000:2.0:1789760716.686580:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493668224 00000100:00100000:1.0:1789760716.686777:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726883043823 time 726958414418 state 0x0 ref 1 00000100:00100000:1.0:1789760716.686793:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726958674834 time 726958414418 state 0x1 ref 1 00000100:00100000:1.0:1789760716.686821:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726958674834 time 727027857668 state 0x0 ref 1 00000100:00100000:1.0:1789760716.686834:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493668224:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760716.686881:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22011904, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa807 00010000:00010000:1.0:1789760716.686890:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0491c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.686905:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760716.686910:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.686916:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0491c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.686929:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0491c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760716.739611:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22011904 -> 22016000 00010000:00010000:1.0:1789760716.739632:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0491c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.739646:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0491c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.739655:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0491c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.739675:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760716.739680:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0491c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.739690:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0491c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.739829:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0491c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.739839:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0491c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.739931:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493668224:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53095us (54147us total) trans 4294972693 rc 0/0 00000100:00100000:1.0:1789760716.739946:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5404 00000100:00100000:1.0:1789760716.739957:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726958674834 time 727027857668 state 0x2 ref 0 00000001:00080000:2.0:1789760716.743556:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972693 is committed 00000100:00100000:1.0:1789760716.747729:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760716.747810:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493668352 00000100:00100000:3.0:1789760716.747819:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493668352 00000100:00100000:3.0:1789760716.747876:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 726958674834 time 727027857668 state 0x0 ref 1 00000100:00100000:3.0:1789760716.747892:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727028118084 time 727027857668 state 0x1 ref 1 00000100:00100000:3.0:1789760716.747923:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727028118084 time 727088958473 state 0x0 ref 1 00000100:00100000:3.0:1789760716.747936:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493668352:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760716.748019:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22016000, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa807 00010000:00010000:3.0:1789760716.748029:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04923 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.748043:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760716.748048:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.748054:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04923 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.748073:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04923 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760716.786542:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972694 is committed 00002000:00010000:3.0:1789760716.786857:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22016000 -> 22020096 00010000:00010000:3.0:1789760716.786872:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04923 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.786884:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04923 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.786893:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04923 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.786900:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760716.786904:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04923 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.786913:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04923 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.786936:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04923 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.786944:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04923 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760716.787071:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493668352:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39133us (39358us total) trans 4294972694 rc 0/0 00000100:00100000:3.0:1789760716.787082:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5405 00000100:00100000:3.0:1789760716.787092:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727028118084 time 727088958473 state 0x2 ref 0 00000100:00100000:2.0:1789760716.792310:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760716.792871:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493668480 00000100:00100000:3.0:1789760716.792882:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493668480 00000100:00100000:3.0:1789760716.792938:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727028118084 time 727088958473 state 0x0 ref 1 00000100:00100000:3.0:1789760716.792955:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727089218889 time 727088958473 state 0x1 ref 1 00000100:00100000:3.0:1789760716.793071:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727089218889 time 727134106294 state 0x0 ref 1 00000100:00100000:3.0:1789760716.793085:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c89f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493668480:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760716.793156:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22020096, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xa807 00010000:00010000:3.0:1789760716.793167:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0492a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.793184:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760716.793190:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.793197:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0492a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.793211:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0492a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760716.840062:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972695 is committed 00002000:00010000:0.0:1789760716.841792:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22020096 -> 22024192 00002000:00010000:0.0:1789760716.841801:0:6694:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb blocks from disk: 43015 -> 45063 00010000:00010000:0.0:1789760716.841814:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0492a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.841828:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0492a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.841838:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0492a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.841845:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760716.841850:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0492a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.841859:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0492a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.841879:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0492a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.841892:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0492a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.842034:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c89f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493668480:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48948us (49730us total) trans 4294972695 rc 0/0 00000100:00100000:0.0:1789760716.842045:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5406 00000100:00100000:0.0:1789760716.842054:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727089218889 time 727134106294 state 0x2 ref 0 00000100:00100000:1.0:1789760716.850902:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760716.851387:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493668608 00000100:00100000:3.0:1789760716.851397:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493668608 00000100:00100000:3.0:1789760716.851451:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727089218889 time 727134106294 state 0x0 ref 1 00000100:00100000:3.0:1789760716.851466:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727134366710 time 727134106294 state 0x1 ref 1 00000100:00100000:3.0:1789760716.851497:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727134366710 time 727192533468 state 0x0 ref 1 00000100:00100000:3.0:1789760716.851509:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493668608:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760716.851560:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22024192, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:3.0:1789760716.851568:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04931 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.851585:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760716.851590:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.851595:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04931 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.851608:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04931 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760716.894926:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22024192 -> 22028288 00010000:00010000:2.0:1789760716.894948:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04931 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.894991:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04931 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.895000:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04931 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.895008:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760716.895013:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04931 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.895022:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04931 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.895041:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04931 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.895048:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04931 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760716.895175:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493668608:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43660us (44287us total) trans 4294972696 rc 0/0 00000100:00100000:2.0:1789760716.895188:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5407 00000100:00100000:2.0:1789760716.895199:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727134366710 time 727192533468 state 0x2 ref 0 00000001:00080000:2.0:1789760716.895242:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972696 is committed 00000100:00100000:2.0:1789760716.899451:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760716.899529:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493668736 00000100:00100000:3.0:1789760716.899539:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493668736 00000100:00100000:3.0:1789760716.899593:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727134366710 time 727192533468 state 0x0 ref 1 00000100:00100000:3.0:1789760716.899609:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727192793884 time 727192533468 state 0x1 ref 1 00000100:00100000:3.0:1789760716.899631:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727192793884 time 727240667027 state 0x0 ref 1 00000100:00100000:3.0:1789760716.899643:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493668736:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760716.899703:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22028288, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:3.0:1789760716.899713:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04938 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.899737:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760716.899741:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760716.899748:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04938 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760716.899761:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04938 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760716.904292:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760716.904341:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760716.904366:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760716.904393:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493668864 00000100:00100000:3.0:1789760716.904403:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493668864 00000100:00100000:3.0:1789760716.904456:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 389 00000100:00100000:3.0:1789760716.904466:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d4e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2370:x1876699493668864:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:3.0:1789760716.904531:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d4e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2370:x1876699493668864:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 66us (189us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760716.904539:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 389 00000100:00100000:3.0:1789760716.904573:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493668992 00000100:00100000:3.0:1789760716.904576:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493668992 00000100:00100000:3.0:1789760716.904608:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493669120 00000100:00100000:3.0:1789760716.904611:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493669120 00000100:00100000:3.0:1789760716.904632:0:9460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 457 00000100:00100000:3.0:1789760716.904638:0:9460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2370:x1876699493668992:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:3.0:1789760716.904677:0:9460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2370:x1876699493668992:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 39us (395us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760716.904683:0:9460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 457 00000100:00100000:3.0:1789760716.904694:0:9460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 458 00000100:00100000:3.0:1789760716.904700:0:9460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2370:x1876699493669120:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:3.0:1789760716.904734:0:9460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2370:x1876699493669120:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 34us (368us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760716.904740:0:9460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 458 00000100:00080000:3.0:1789760716.904758:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:0.0:1789760716.942457:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972697 is committed 00002000:00010000:2.0:1789760716.942820:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22028288 -> 22032384 00010000:00010000:2.0:1789760716.942836:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04938 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.942850:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04938 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.942860:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04938 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.942868:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760716.942873:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04938 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.942884:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04938 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.942904:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04938 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.942913:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04938 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760716.944111:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493668736:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44460us (44658us total) trans 4294972697 rc 0/0 00000100:00100000:2.0:1789760716.944126:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5408 00000100:00100000:2.0:1789760716.944137:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727192793884 time 727240667027 state 0x2 ref 0 00000100:00100000:3.0:1789760716.947574:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760716.947681:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493669248 00000100:00100000:0.0:1789760716.947692:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493669248 00000100:00100000:0.0:1789760716.947754:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727192793884 time 727240667027 state 0x0 ref 1 00000100:00100000:0.0:1789760716.947768:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727240927443 time 727240667027 state 0x1 ref 1 00000100:00100000:0.0:1789760716.947797:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727240927443 time 727288832588 state 0x0 ref 1 00000100:00100000:0.0:1789760716.947809:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493669248:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760716.947856:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22032384, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:0.0:1789760716.947864:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0493f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.947879:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760716.947883:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760716.947890:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0493f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.947903:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0493f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760716.974647:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972698 is committed 00002000:00010000:0.0:1789760716.974912:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22032384 -> 22036480 00010000:00010000:0.0:1789760716.974927:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0493f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.974940:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0493f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.974949:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0493f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.974956:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760716.975018:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0493f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.975028:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0493f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.975047:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0493f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.975054:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0493f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.975313:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493669248:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27500us (27744us total) trans 4294972698 rc 0/0 00000100:00100000:0.0:1789760716.975327:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5409 00000100:00100000:0.0:1789760716.975337:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727240927443 time 727288832588 state 0x2 ref 0 00000100:00100000:2.0:1789760716.978839:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760716.979319:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493669376 00000100:00100000:1.0:1789760716.979330:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493669376 00000100:00100000:1.0:1789760716.979385:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727240927443 time 727288832588 state 0x0 ref 1 00000100:00100000:1.0:1789760716.979401:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727289093004 time 727288832588 state 0x1 ref 1 00000100:00100000:1.0:1789760716.979431:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727289093004 time 727320465906 state 0x0 ref 1 00000100:00100000:1.0:1789760716.979444:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493669376:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760716.979495:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22036480, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:1.0:1789760716.979505:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04946 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.979521:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760716.979527:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.979534:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04946 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.979548:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04946 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760717.023480:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22036480 -> 22040576 00010000:00010000:2.0:1789760717.023500:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04946 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.023513:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04946 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.023523:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04946 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.023531:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760717.023536:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04946 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.023546:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04946 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.023566:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04946 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.023575:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04946 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760717.023685:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493669376:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44237us (44851us total) trans 4294972699 rc 0/0 00000100:00100000:2.0:1789760717.023706:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5410 00000100:00100000:2.0:1789760717.023722:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727289093004 time 727320465906 state 0x2 ref 0 00000001:00080000:3.0:1789760717.023863:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972699 is committed 00000100:00100000:2.0:1789760717.027645:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760717.027793:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493669504 00000100:00100000:3.0:1789760717.027803:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493669504 00000100:00100000:3.0:1789760717.027881:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727289093004 time 727320465906 state 0x0 ref 1 00000100:00100000:3.0:1789760717.027896:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727320726322 time 727320465906 state 0x1 ref 1 00000100:00100000:3.0:1789760717.027928:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727320726322 time 727368963332 state 0x0 ref 1 00000100:00100000:3.0:1789760717.027941:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc17480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493669504:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760717.028023:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22040576, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:3.0:1789760717.028032:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0494d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.028046:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760717.028051:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760717.028057:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0494d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.028070:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0494d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760717.028644:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760717.094692:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972700 is committed 00002000:00010000:3.0:1789760717.094909:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22040576 -> 22044672 00010000:00010000:3.0:1789760717.094926:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0494d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.094937:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0494d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.094945:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0494d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.094952:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760717.094995:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0494d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.095005:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0494d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.095024:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0494d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.095032:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0494d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760717.095126:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc17480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493669504:12345-192.168.203.60@tcp:4:dd.0 Request processed in 67182us (67488us total) trans 4294972700 rc 0/0 00000100:00100000:3.0:1789760717.095136:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5411 00000100:00100000:3.0:1789760717.095146:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727320726322 time 727368963332 state 0x2 ref 0 00000100:00100000:0.0:1789760717.098363:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760717.098460:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493669632 00000100:00100000:2.0:1789760717.098470:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493669632 00000100:00100000:2.0:1789760717.098526:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727320726322 time 727368963332 state 0x0 ref 1 00000100:00100000:2.0:1789760717.098544:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727369223748 time 727368963332 state 0x1 ref 1 00000100:00100000:2.0:1789760717.098627:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727369223748 time 727439661959 state 0x0 ref 1 00000100:00100000:2.0:1789760717.098639:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493669632:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760717.098696:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22044672, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:2.0:1789760717.098706:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04954 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.098722:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760717.098726:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760717.098733:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04954 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.098748:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04954 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760717.131675:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972701 is committed 00002000:00010000:1.0:1789760717.131897:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22044672 -> 22048768 00010000:00010000:1.0:1789760717.131913:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04954 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.131925:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04954 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.131934:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04954 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.131941:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760717.131946:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04954 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.131955:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04954 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.132007:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04954 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.132015:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04954 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760717.132097:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493669632:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33455us (33739us total) trans 4294972701 rc 0/0 00000100:00100000:1.0:1789760717.132108:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5412 00000100:00100000:1.0:1789760717.132117:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727369223748 time 727439661959 state 0x2 ref 0 00000100:00100000:1.0:1789760717.136322:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760717.136421:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493669760 00000100:00100000:2.0:1789760717.136430:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493669760 00000100:00100000:2.0:1789760717.136487:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727369223748 time 727439661959 state 0x0 ref 1 00000100:00100000:2.0:1789760717.136503:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727439922375 time 727439661959 state 0x1 ref 1 00000100:00100000:2.0:1789760717.136575:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727439922375 time 727477610107 state 0x0 ref 1 00000100:00100000:2.0:1789760717.136589:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493669760:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760717.136635:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22048768, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:2.0:1789760717.136644:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0495b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.136671:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760717.136677:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760717.136684:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0495b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.136699:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0495b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760717.172878:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22048768 -> 22052864 00010000:00010000:1.0:1789760717.172897:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0495b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.172909:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0495b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.172917:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0495b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.172925:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760717.172930:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0495b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.172940:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0495b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.172957:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0495b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.172994:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0495b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760717.173081:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493669760:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36490us (36766us total) trans 4294972702 rc 0/0 00000100:00100000:1.0:1789760717.173091:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5413 00000100:00100000:1.0:1789760717.173101:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727439922375 time 727477610107 state 0x2 ref 0 00000001:00080000:2.0:1789760717.173306:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972702 is committed 00000100:00100000:2.0:1789760717.176062:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760717.176237:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493669888 00000100:00100000:1.0:1789760717.176247:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493669888 00000100:00100000:1.0:1789760717.176304:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727439922375 time 727477610107 state 0x0 ref 1 00000100:00100000:1.0:1789760717.176320:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727477870523 time 727477610107 state 0x1 ref 1 00000100:00100000:1.0:1789760717.176350:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727477870523 time 727517385692 state 0x0 ref 1 00000100:00100000:1.0:1789760717.176362:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493669888:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760717.176409:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22052864, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:1.0:1789760717.176419:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04962 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.176433:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760717.176438:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760717.176444:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04962 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.176457:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04962 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760717.208645:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972703 is committed 00002000:00010000:2.0:1789760717.208821:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22052864 -> 22056960 00010000:00010000:2.0:1789760717.208836:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04962 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.208855:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04962 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.208865:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04962 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.208873:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760717.208877:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04962 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.208887:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04962 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.208907:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04962 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.208914:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04962 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760717.209055:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493669888:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32688us (32997us total) trans 4294972703 rc 0/0 00000100:00100000:2.0:1789760717.209066:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5414 00000100:00100000:2.0:1789760717.209076:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727477870523 time 727517385692 state 0x2 ref 0 00000100:00100000:0.0:1789760717.215992:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760717.216082:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493670016 00000100:00100000:3.0:1789760717.216092:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493670016 00000100:00100000:3.0:1789760717.216221:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727477870523 time 727517385692 state 0x0 ref 1 00000100:00100000:3.0:1789760717.216236:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727517646108 time 727517385692 state 0x1 ref 1 00000100:00100000:3.0:1789760717.216397:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727517646108 time 727557430585 state 0x0 ref 1 00000100:00100000:3.0:1789760717.216413:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493670016:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760717.216467:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22056960, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:3.0:1789760717.216477:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04969 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.216493:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760717.216497:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760717.216504:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04969 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.216518:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04969 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760717.248688:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972704 is committed 00002000:00010000:2.0:1789760717.248916:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22056960 -> 22061056 00010000:00010000:2.0:1789760717.248931:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04969 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.248943:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04969 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.248952:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04969 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.248990:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760717.248996:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04969 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.249005:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04969 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.249025:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04969 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.249033:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04969 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760717.249128:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493670016:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32715us (33143us total) trans 4294972704 rc 0/0 00000100:00100000:2.0:1789760717.249221:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5415 00000100:00100000:2.0:1789760717.249233:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727517646108 time 727557430585 state 0x2 ref 0 00000100:00100000:1.0:1789760717.252537:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760717.252621:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493670144 00000100:00100000:2.0:1789760717.252631:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493670144 00000100:00100000:2.0:1789760717.252695:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727517646108 time 727557430585 state 0x0 ref 1 00000100:00100000:2.0:1789760717.252719:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727557691001 time 727557430585 state 0x1 ref 1 00000100:00100000:2.0:1789760717.252758:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727557691001 time 727593793433 state 0x0 ref 1 00000100:00100000:2.0:1789760717.252772:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493670144:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760717.252821:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22061056, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:2.0:1789760717.252830:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04970 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.252847:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760717.252852:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760717.252858:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04970 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.252872:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04970 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760717.324437:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972705 is committed 00002000:00010000:3.0:1789760717.324537:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22061056 -> 22065152 00010000:00010000:3.0:1789760717.324552:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04970 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.324563:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04970 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.324571:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04970 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.324578:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760717.324582:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04970 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.324591:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04970 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.324610:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04970 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.324617:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04970 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760717.324849:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493670144:12345-192.168.203.60@tcp:4:dd.0 Request processed in 72074us (72314us total) trans 4294972705 rc 0/0 00000100:00100000:3.0:1789760717.324859:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5416 00000100:00100000:3.0:1789760717.324868:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727557691001 time 727593793433 state 0x2 ref 0 00000100:00100000:0.0:1789760717.333743:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760717.333839:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493670272 00000100:00100000:1.0:1789760717.333849:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493670272 00000100:00100000:1.0:1789760717.333906:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727557691001 time 727593793433 state 0x0 ref 1 00000100:00100000:1.0:1789760717.333922:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727594053849 time 727593793433 state 0x1 ref 1 00000100:00100000:1.0:1789760717.333954:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727594053849 time 727674988969 state 0x0 ref 1 00000100:00100000:1.0:1789760717.334001:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493670272:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760717.334050:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22065152, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:1.0:1789760717.334058:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04977 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.334072:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760717.334076:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760717.334082:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04977 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.334096:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04977 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.371070:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760717.383337:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972706 is committed 00002000:00010000:0.0:1789760717.383542:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22065152 -> 22069248 00010000:00010000:0.0:1789760717.383557:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04977 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.383570:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04977 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.383579:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04977 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.383586:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760717.383591:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04977 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.383600:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04977 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.383620:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04977 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.383630:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04977 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.383732:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493670272:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49729us (49994us total) trans 4294972706 rc 0/0 00000100:00100000:0.0:1789760717.383743:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5417 00000100:00100000:0.0:1789760717.383752:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727594053849 time 727674988969 state 0x2 ref 0 00000100:00100000:0.0:1789760717.387080:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760717.387168:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493670400 00000100:00100000:1.0:1789760717.387179:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493670400 00000100:00100000:1.0:1789760717.387236:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727594053849 time 727674988969 state 0x0 ref 1 00000100:00100000:1.0:1789760717.387252:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727675249385 time 727674988969 state 0x1 ref 1 00000100:00100000:1.0:1789760717.387281:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727675249385 time 727728316660 state 0x0 ref 1 00000100:00100000:1.0:1789760717.387293:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493670400:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760717.387345:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22069248, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:1.0:1789760717.387354:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0497e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.387370:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760717.387374:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760717.387380:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0497e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.387395:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0497e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760717.421029:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972707 is committed 00002000:00010000:2.0:1789760717.421250:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22069248 -> 22073344 00010000:00010000:2.0:1789760717.421268:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0497e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.421282:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0497e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.421293:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0497e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.421302:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760717.421308:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0497e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.421319:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0497e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.421339:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0497e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.421349:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0497e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760717.421482:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493670400:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34186us (34406us total) trans 4294972707 rc 0/0 00000100:00100000:2.0:1789760717.421493:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5418 00000100:00100000:2.0:1789760717.421503:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727675249385 time 727728316660 state 0x2 ref 0 00000100:00100000:1.0:1789760717.425407:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760717.425495:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493670528 00000100:00100000:3.0:1789760717.425505:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493670528 00000100:00100000:3.0:1789760717.425559:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727675249385 time 727728316660 state 0x0 ref 1 00000100:00100000:3.0:1789760717.425573:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727728577076 time 727728316660 state 0x1 ref 1 00000100:00100000:3.0:1789760717.425645:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727728577076 time 727766679330 state 0x0 ref 1 00000100:00100000:3.0:1789760717.425667:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493670528:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760717.425725:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22073344, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:3.0:1789760717.425735:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04985 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.425751:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760717.425755:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760717.425761:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04985 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.425774:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04985 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760717.459408:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972708 is committed 00002000:00010000:3.0:1789760717.459547:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22073344 -> 22077440 00010000:00010000:3.0:1789760717.459563:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04985 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.459576:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04985 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.459586:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04985 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.459593:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760717.459598:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04985 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.459608:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04985 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.459633:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04985 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.459641:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04985 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760717.463441:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493670528:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37773us (38029us total) trans 4294972708 rc 0/0 00000100:00100000:3.0:1789760717.463457:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5419 00000100:00100000:3.0:1789760717.463468:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727728577076 time 727766679330 state 0x2 ref 0 00000100:00100000:0.0:1789760717.467214:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760717.467344:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493670656 00000100:00100000:3.0:1789760717.467354:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493670656 00000100:00100000:3.0:1789760717.467408:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727728577076 time 727766679330 state 0x0 ref 1 00000100:00100000:3.0:1789760717.467424:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727766939746 time 727766679330 state 0x1 ref 1 00000100:00100000:3.0:1789760717.467456:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727766939746 time 727808491717 state 0x0 ref 1 00000100:00100000:3.0:1789760717.467481:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493670656:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760717.467535:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22077440, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:3.0:1789760717.467545:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0498c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.467561:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760717.467566:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760717.467572:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0498c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.467586:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0498c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760717.516802:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972709 is committed 00002000:00010000:3.0:1789760717.517005:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22077440 -> 22081536 00010000:00010000:3.0:1789760717.517021:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0498c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.517035:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0498c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.517045:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0498c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.517052:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760717.517057:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0498c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.517066:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0498c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.517152:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0498c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.517162:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0498c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760717.517250:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493670656:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49767us (50042us total) trans 4294972709 rc 0/0 00000100:00100000:3.0:1789760717.517260:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5420 00000100:00100000:3.0:1789760717.517269:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727766939746 time 727808491717 state 0x2 ref 0 00000100:00100000:0.0:1789760717.526477:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760717.527253:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493670784 00000100:00100000:1.0:1789760717.527263:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493670784 00000100:00100000:1.0:1789760717.527325:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727766939746 time 727808491717 state 0x0 ref 1 00000100:00100000:1.0:1789760717.527341:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727808752133 time 727808491717 state 0x1 ref 1 00000100:00100000:1.0:1789760717.527379:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727808752133 time 727868413852 state 0x0 ref 1 00000100:00100000:1.0:1789760717.527392:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493670784:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760717.527440:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22081536, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:1.0:1789760717.527449:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04993 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.527465:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760717.527470:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760717.527476:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04993 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.527490:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04993 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760717.589547:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972710 is committed 00002000:00010000:2.0:1789760717.590079:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22081536 -> 22085632 00010000:00010000:2.0:1789760717.590096:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04993 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.590109:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04993 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.590117:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04993 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.590126:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760717.590131:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04993 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.590208:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04993 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.590229:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04993 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.590236:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04993 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760717.590323:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493670784:12345-192.168.203.60@tcp:4:dd.0 Request processed in 62929us (63853us total) trans 4294972710 rc 0/0 00000100:00100000:2.0:1789760717.590335:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5421 00000100:00100000:2.0:1789760717.590345:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727808752133 time 727868413852 state 0x2 ref 0 00000100:00100000:1.0:1789760717.594519:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760717.594610:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493670912 00000100:00100000:2.0:1789760717.594620:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493670912 00000100:00100000:2.0:1789760717.594686:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727808752133 time 727868413852 state 0x0 ref 1 00000100:00100000:2.0:1789760717.594703:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727868674268 time 727868413852 state 0x1 ref 1 00000100:00100000:2.0:1789760717.594823:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727868674268 time 727935857753 state 0x0 ref 1 00000100:00100000:2.0:1789760717.594837:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493670912:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760717.594885:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22085632, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0xb007 00010000:00010000:2.0:1789760717.594893:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0499a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.594909:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760717.594913:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760717.594920:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0499a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.594935:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0499a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760717.632336:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972711 is committed 00002000:00010000:2.0:1789760717.632512:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22085632 -> 22089728 00002000:00010000:2.0:1789760717.632517:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760716 -> 1789760717 00002000:00010000:2.0:1789760717.632519:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760716 -> 1789760717 00010000:00010000:2.0:1789760717.632532:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0499a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.632550:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0499a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.632558:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0499a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.632565:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760717.632570:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0499a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.632578:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0499a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.632596:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0499a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.632604:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0499a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760717.632806:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493670912:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37966us (38291us total) trans 4294972711 rc 0/0 00000100:00100000:2.0:1789760717.632818:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5422 00000100:00100000:2.0:1789760717.632828:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727868674268 time 727935857753 state 0x2 ref 0 00000100:00100000:0.0:1789760717.637501:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760717.637598:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493671040 00000100:00100000:2.0:1789760717.637609:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493671040 00000100:00100000:2.0:1789760717.637677:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727868674268 time 727935857753 state 0x0 ref 1 00000100:00100000:2.0:1789760717.637693:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727936118169 time 727935857753 state 0x1 ref 1 00000100:00100000:2.0:1789760717.637827:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727936118169 time 727978861014 state 0x0 ref 1 00000100:00100000:2.0:1789760717.637842:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493671040:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760717.637892:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22089728, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:2.0:1789760717.637902:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e049a1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.637918:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760717.637923:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760717.637929:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e049a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.637943:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e049a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760717.690335:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760717.690356:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620ffe84e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3304:1876699519248256:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760717.690430:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760717.690503:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519248256 00000100:00100000:1.0:1789760717.690510:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519248256 00000100:00100000:1.0:1789760717.690560:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 190 00000100:00100000:1.0:1789760717.690569:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519248256:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760717.690592:0:6690:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528547840 pending 1703936 free 3836739584 avail 3814719488 00000100:00100000:1.0:1789760717.690634:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519248256:12345-0@lo:13:osp-pre-0-0.0 Request processed in 66us (204us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760717.690645:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 190 00000100:00080000:1.0:1789760717.690677:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760717.690724:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620ffe84e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3304:1876699519248256:0@lo:13:osp-pre-0-0.0 00002000:00010000:2.0:1789760717.691746:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22089728 -> 22093824 00010000:00010000:2.0:1789760717.691762:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e049a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.691774:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e049a1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.691788:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e049a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.691797:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760717.691801:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e049a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.691810:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e049a1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.691832:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e049a1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.691841:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e049a1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760717.691929:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493671040:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54087us (54435us total) trans 4294972712 rc 0/0 00000100:00100000:2.0:1789760717.691940:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5423 00000100:00100000:2.0:1789760717.691950:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727936118169 time 727978861014 state 0x2 ref 0 00000001:00080000:1.0:1789760717.692795:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972712 is committed 00000100:00100000:2.0:1789760717.695704:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760717.695791:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493671168 00000100:00100000:3.0:1789760717.695800:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493671168 00000100:00100000:3.0:1789760717.695853:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727936118169 time 727978861014 state 0x0 ref 1 00000100:00100000:3.0:1789760717.695869:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727979121430 time 727978861014 state 0x1 ref 1 00000100:00100000:3.0:1789760717.695906:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727979121430 time 728036941771 state 0x0 ref 1 00000100:00100000:3.0:1789760717.695920:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493671168:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760717.696005:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22093824, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:3.0:1789760717.696014:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049a8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.696030:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760717.696035:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760717.696041:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.696054:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760717.731869:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22093824 -> 22097920 00010000:00010000:0.0:1789760717.731902:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.731916:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049a8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.731926:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.731933:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760717.731939:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.731948:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049a8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.731998:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049a8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.732007:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049a8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.735727:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493671168:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36171us (36395us total) trans 4294972713 rc 0/0 00000100:00100000:0.0:1789760717.735742:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5424 00000100:00100000:0.0:1789760717.735752:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727979121430 time 728036941771 state 0x2 ref 0 00000100:00100000:1.0:1789760717.736559:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760717.736640:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493671296 00000100:00100000:3.0:1789760717.736650:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493671296 00000100:00100000:3.0:1789760717.736861:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 727979121430 time 728036941771 state 0x0 ref 1 00000100:00100000:3.0:1789760717.736947:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728037202187 time 728036941771 state 0x1 ref 1 00000100:00100000:3.0:1789760717.737045:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728037202187 time 728078079828 state 0x0 ref 1 00000100:00100000:3.0:1789760717.737058:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493671296:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760717.737239:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22097920, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:3.0:1789760717.737249:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e049af lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.737263:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760717.737267:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760717.737273:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e049af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.737285:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e049af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760717.738549:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972713 is committed 00000001:00080000:2.0:1789760717.776605:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972714 is committed 00002000:00010000:0.0:1789760717.776718:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22097920 -> 22102016 00010000:00010000:0.0:1789760717.776734:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e049af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.776746:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e049af lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.776754:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e049af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.776760:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760717.776765:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e049af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.776775:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e049af lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.776795:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e049af lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.776802:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e049af lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.776899:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493671296:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39839us (40341us total) trans 4294972714 rc 0/0 00000100:00100000:0.0:1789760717.776910:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5425 00000100:00100000:0.0:1789760717.776920:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728037202187 time 728078079828 state 0x2 ref 0 00000100:00100000:1.0:1789760717.781788:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760717.781894:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493671424 00000100:00100000:2.0:1789760717.781905:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493671424 00000100:00100000:2.0:1789760717.781979:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728037202187 time 728078079828 state 0x0 ref 1 00000100:00100000:2.0:1789760717.781994:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728078340244 time 728078079828 state 0x1 ref 1 00000100:00100000:2.0:1789760717.782027:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728078340244 time 728123062773 state 0x0 ref 1 00000100:00100000:2.0:1789760717.782039:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493671424:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760717.782094:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22102016, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:2.0:1789760717.782104:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e049b6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.782119:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760717.782124:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760717.782130:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e049b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.782386:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e049b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760717.822421:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22102016 -> 22106112 00010000:00010000:1.0:1789760717.822443:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e049b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.822456:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e049b6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.822466:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e049b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.822473:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760717.822478:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e049b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.822486:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e049b6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.822511:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e049b6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.822525:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e049b6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760717.822621:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493671424:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40578us (40838us total) trans 4294972715 rc 0/0 00000100:00100000:1.0:1789760717.822633:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5426 00000100:00100000:1.0:1789760717.822644:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728078340244 time 728123062773 state 0x2 ref 0 00000001:00080000:3.0:1789760717.823894:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972715 is committed 00000100:00100000:2.0:1789760717.827862:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760717.828273:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493671552 00000100:00100000:0.0:1789760717.828283:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493671552 00000100:00100000:1.0:1789760717.828422:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728078340244 time 728123062773 state 0x0 ref 1 00000100:00100000:1.0:1789760717.828438:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728123323189 time 728123062773 state 0x1 ref 1 00000100:00100000:1.0:1789760717.828499:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728123323189 time 728169534453 state 0x0 ref 1 00000100:00100000:1.0:1789760717.828512:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493671552:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760717.828558:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22106112, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:1.0:1789760717.828567:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e049bd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.828581:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760717.828585:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760717.828591:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e049bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.828604:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e049bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760717.900023:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972716 is committed 00002000:00010000:1.0:1789760717.900313:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22106112 -> 22110208 00010000:00010000:1.0:1789760717.900329:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e049bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.900341:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e049bd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.900349:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e049bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.900356:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760717.900361:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e049bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.900371:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e049bd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.900391:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e049bd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.900399:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e049bd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760717.900493:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493671552:12345-192.168.203.60@tcp:4:dd.0 Request processed in 71978us (72640us total) trans 4294972716 rc 0/0 00000100:00100000:1.0:1789760717.900507:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5427 00000100:00100000:1.0:1789760717.900517:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728123323189 time 728169534453 state 0x2 ref 0 00000100:00100000:2.0:1789760717.905211:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760717.905321:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493671680 00000100:00100000:3.0:1789760717.905331:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493671680 00000100:00100000:3.0:1789760717.905388:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728123323189 time 728169534453 state 0x0 ref 1 00000100:00100000:3.0:1789760717.905407:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728169794869 time 728169534453 state 0x1 ref 1 00000100:00100000:3.0:1789760717.905467:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728169794869 time 728246502343 state 0x0 ref 1 00000100:00100000:3.0:1789760717.905481:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493671680:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760717.905529:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22110208, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:3.0:1789760717.905538:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e049c4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.905555:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760717.905560:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760717.905568:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e049c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.905582:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e049c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760717.946360:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972717 is committed 00002000:00010000:2.0:1789760717.946428:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22110208 -> 22114304 00010000:00010000:2.0:1789760717.946444:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e049c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.946463:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e049c4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.946472:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e049c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.946480:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760717.946484:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e049c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.946493:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e049c4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.946511:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e049c4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.946519:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e049c4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760717.946850:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493671680:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41366us (41642us total) trans 4294972717 rc 0/0 00000100:00100000:2.0:1789760717.946861:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5428 00000100:00100000:2.0:1789760717.947869:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728169794869 time 728246502343 state 0x2 ref 0 00000100:00100000:0.0:1789760717.951991:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760717.952114:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493671808 00000100:00100000:3.0:1789760717.952123:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493671808 00000100:00100000:3.0:1789760717.952184:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728169794869 time 728246502343 state 0x0 ref 1 00000100:00100000:3.0:1789760717.952197:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728246762759 time 728246502343 state 0x1 ref 1 00000100:00100000:3.0:1789760717.952275:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728246762759 time 728293309585 state 0x0 ref 1 00000100:00100000:3.0:1789760717.952287:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493671808:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760717.952336:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22114304, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:3.0:1789760717.952343:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e049cb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.952357:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760717.952361:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760717.952367:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e049cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760717.952381:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e049cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760717.997428:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22114304 -> 22118400 00010000:00010000:0.0:1789760717.997446:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e049cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.997458:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e049cb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.997467:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e049cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.997474:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760717.997478:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e049cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.997487:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e049cb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.997505:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e049cb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.997513:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e049cb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.997610:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493671808:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45320us (45654us total) trans 4294972718 rc 0/0 00000100:00100000:0.0:1789760717.997620:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5429 00000100:00100000:0.0:1789760717.997630:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728246762759 time 728293309585 state 0x2 ref 0 00000001:00080000:2.0:1789760717.997718:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972718 is committed 00000100:00100000:0.0:1789760718.001619:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760718.001722:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493671936 00000100:00100000:1.0:1789760718.001732:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493671936 00000100:00100000:1.0:1789760718.001785:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728246762759 time 728293309585 state 0x0 ref 1 00000100:00100000:1.0:1789760718.001801:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728293570001 time 728293309585 state 0x1 ref 1 00000100:00100000:1.0:1789760718.001831:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728293570001 time 728342866705 state 0x0 ref 1 00000100:00100000:1.0:1789760718.001843:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493671936:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760718.002094:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22118400, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:1.0:1789760718.002105:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e049d2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.002122:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760718.002127:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.002252:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e049d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.002269:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e049d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760718.045673:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972719 is committed 00002000:00010000:3.0:1789760718.045843:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22118400 -> 22122496 00010000:00010000:3.0:1789760718.045859:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e049d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.045882:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e049d2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.045891:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e049d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.045899:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760718.045903:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e049d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.045911:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e049d2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.045930:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e049d2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.045937:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e049d2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.046065:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493671936:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44218us (44451us total) trans 4294972719 rc 0/0 00000100:00100000:3.0:1789760718.046075:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5430 00000100:00100000:3.0:1789760718.046093:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728293570001 time 728342866705 state 0x2 ref 0 00000100:00100000:1.0:1789760718.050369:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760718.050529:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493672064 00000100:00100000:3.0:1789760718.050540:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493672064 00000100:00100000:3.0:1789760718.050596:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728293570001 time 728342866705 state 0x0 ref 1 00000100:00100000:3.0:1789760718.050612:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728343127121 time 728342866705 state 0x1 ref 1 00000100:00100000:3.0:1789760718.050689:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728343127121 time 728391723363 state 0x0 ref 1 00000100:00100000:3.0:1789760718.050702:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493672064:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760718.050820:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22122496, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:3.0:1789760718.050830:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049d9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.050844:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760718.050850:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.050857:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.051024:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760718.123679:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22122496 -> 22126592 00010000:00010000:3.0:1789760718.123700:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.123724:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049d9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.123734:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.123741:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760718.123746:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.123755:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049d9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.123775:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049d9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.123789:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049d9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.123891:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493672064:12345-192.168.203.60@tcp:4:dd.0 Request processed in 73187us (73526us total) trans 4294972720 rc 0/0 00000100:00100000:3.0:1789760718.123901:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5431 00000100:00100000:3.0:1789760718.123911:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728343127121 time 728391723363 state 0x2 ref 0 00000100:00100000:0.0:1789760718.127804:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760718.127894:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493672192 00000100:00100000:3.0:1789760718.127903:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493672192 00000100:00100000:3.0:1789760718.128000:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728343127121 time 728391723363 state 0x0 ref 1 00000100:00100000:3.0:1789760718.128018:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728391983779 time 728391723363 state 0x1 ref 1 00000100:00100000:3.0:1789760718.128044:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728391983779 time 728469079277 state 0x0 ref 1 00000100:00100000:3.0:1789760718.128056:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493672192:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760718.128116:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22126592, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:3.0:1789760718.128126:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e049e0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.128152:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760718.128156:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.128161:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e049e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.128180:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e049e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760718.133005:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972720 is committed 00000001:00080000:2.0:1789760718.180468:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972721 is committed 00002000:00010000:3.0:1789760718.180584:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22126592 -> 22130688 00010000:00010000:3.0:1789760718.180600:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e049e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.180616:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e049e0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.180626:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e049e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.180634:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760718.180640:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e049e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.180649:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e049e0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.180685:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e049e0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.180692:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e049e0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.180835:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493672192:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52776us (53035us total) trans 4294972721 rc 0/0 00000100:00100000:3.0:1789760718.180845:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5432 00000100:00100000:3.0:1789760718.180856:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728391983779 time 728469079277 state 0x2 ref 0 00000100:00100000:0.0:1789760718.186015:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760718.186230:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493672320 00000100:00100000:3.0:1789760718.186242:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493672320 00000100:00100000:3.0:1789760718.186296:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728391983779 time 728469079277 state 0x0 ref 1 00000100:00100000:3.0:1789760718.186310:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728469339693 time 728469079277 state 0x1 ref 1 00000100:00100000:3.0:1789760718.186385:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728469339693 time 728527419944 state 0x0 ref 1 00000100:00100000:3.0:1789760718.186398:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493672320:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760718.186447:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22130688, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:3.0:1789760718.186456:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049e7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.186471:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760718.186475:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.186481:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.186495:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.202843:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620ffb8a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3304:1876699519248384:0@lo:400:kworker.0 00000100:00100000:3.0:1789760718.202912:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760718.203393:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519248384 00000100:00100000:0.0:1789760718.203402:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519248384 00000100:00100000:0.0:1789760718.203457:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 390 00000100:00100000:0.0:1789760718.203466:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d4d050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3304:x1876699519248384:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760718.203683:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9621018a1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3303:1876699519248768:0@lo:400:kworker.0 00000100:00100000:1.0:1789760718.203727:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760718.203755:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760718.203761:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9621018a0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3303:1876699519248896:0@lo:400:kworker.0 00000100:00100000:1.0:1789760718.203786:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760718.203814:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620ffb88700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3305:1876699519248512:0@lo:400:kworker.0 00000100:00100000:1.0:1789760718.203843:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760718.203861:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760718.203867:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620ffb89180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3305:1876699519248640:0@lo:400:kworker.0 00000100:00100000:1.0:1789760718.203887:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760718.203915:0:10475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519248896 00000100:00100000:1.0:1789760718.203919:0:10475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519248896 00000100:00100000:1.0:1789760718.203956:0:10475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 168 00000100:00100000:1.0:1789760718.203980:0:10475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc352c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876699519248896:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760718.204019:0:10475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876699519248896:12345-0@lo:400:kworker.0 Request processed in 40us (231us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760718.204025:0:10475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 168 00000100:00100000:1.0:1789760718.204044:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519248768 00000100:00100000:1.0:1789760718.204047:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519248768 00000100:00100000:1.0:1789760718.204069:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519248512 00000100:00100000:1.0:1789760718.204072:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519248512 00000100:00100000:2.0:1789760718.204228:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519248640 00000100:00100000:2.0:1789760718.204234:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519248640 00000100:00100000:2.0:1789760718.204262:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 459 00000100:00100000:2.0:1789760718.204269:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc352d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876699519248768:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760718.204304:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876699519248768:12345-0@lo:400:kworker.0 Request processed in 37us (576us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760718.204310:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 459 00000100:00100000:2.0:1789760718.204322:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 460 00000100:00100000:2.0:1789760718.204329:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc352f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3305:x1876699519248640:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760718.204361:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3305:x1876699519248640:12345-0@lo:400:kworker.0 Request processed in 32us (472us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760718.204366:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 460 00000100:00100000:1.0:1789760718.204430:0:9460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 461 00000100:00100000:1.0:1789760718.204438:0:9460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc352f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3305:x1876699519248512:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760718.204467:0:9460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3305:x1876699519248512:12345-0@lo:400:kworker.0 Request processed in 31us (624us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760718.204473:0:9460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 461 00000100:00080000:1.0:1789760718.204492:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789760718.204496:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789760718.204527:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9621018a0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3303:1876699519248896:0@lo:400:kworker.0 00000100:00100000:1.0:1789760718.204538:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9621018a1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3303:1876699519248768:0@lo:400:kworker.0 00000100:00100000:1.0:1789760718.204565:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620ffb89180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3305:1876699519248640:0@lo:400:kworker.0 00000100:00100000:1.0:1789760718.204573:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620ffb88700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3305:1876699519248512:0@lo:400:kworker.0 00000100:00100000:0.0:1789760718.204693:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d4d050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3304:x1876699519248384:12345-0@lo:400:kworker.0 Request processed in 1227us (1780us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760718.204700:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 390 00000100:00100000:3.0:1789760718.206801:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620ffb8a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3304:1876699519248384:0@lo:400:kworker.0 00000001:00080000:2.0:1789760718.228476:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972722 is committed 00002000:00010000:3.0:1789760718.228586:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22130688 -> 22134784 00010000:00010000:3.0:1789760718.228602:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.228614:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049e7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.228623:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.228631:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760718.228636:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.228645:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049e7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.228676:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049e7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.228682:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e049e7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.228837:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493672320:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42436us (42884us total) trans 4294972722 rc 0/0 00000100:00100000:3.0:1789760718.228849:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5433 00000100:00100000:3.0:1789760718.228859:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728469339693 time 728527419944 state 0x2 ref 0 00000100:00100000:2.0:1789760718.233054:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760718.233239:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493672448 00000100:00100000:3.0:1789760718.233250:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493672448 00000100:00100000:3.0:1789760718.233303:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728469339693 time 728527419944 state 0x0 ref 1 00000100:00100000:3.0:1789760718.233318:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728527680360 time 728527419944 state 0x1 ref 1 00000100:00100000:3.0:1789760718.233343:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728527680360 time 728574378440 state 0x0 ref 1 00000100:00100000:3.0:1789760718.233355:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493672448:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760718.233403:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22134784, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:3.0:1789760718.233412:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e049ee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.233428:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760718.233433:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.233439:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e049ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.233453:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e049ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760718.287338:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22134784 -> 22138880 00010000:00010000:3.0:1789760718.287361:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e049ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.287375:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e049ee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.287384:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e049ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.287391:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760718.287396:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e049ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.287404:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e049ee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.287422:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e049ee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.287429:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e049ee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.287525:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493672448:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54166us (54475us total) trans 4294972723 rc 0/0 00000100:00100000:3.0:1789760718.287540:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5434 00000100:00100000:3.0:1789760718.287550:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728527680360 time 728574378440 state 0x2 ref 0 00000001:00080000:2.0:1789760718.290485:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972723 is committed 00000100:00100000:0.0:1789760718.291461:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760718.291547:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493672576 00000100:00100000:1.0:1789760718.291556:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493672576 00000100:00100000:1.0:1789760718.291615:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728527680360 time 728574378440 state 0x0 ref 1 00000100:00100000:1.0:1789760718.291630:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728574638856 time 728574378440 state 0x1 ref 1 00000100:00100000:1.0:1789760718.291692:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728574638856 time 728632727235 state 0x0 ref 1 00000100:00100000:1.0:1789760718.291704:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493672576:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760718.291756:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22138880, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:1.0:1789760718.291765:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e049f5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.291781:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760718.291785:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.291791:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e049f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.291804:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e049f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760718.335358:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972724 is committed 00002000:00010000:1.0:1789760718.335523:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22138880 -> 22142976 00010000:00010000:1.0:1789760718.335538:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e049f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.335551:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e049f5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.335559:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e049f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.335566:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760718.335571:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e049f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.335580:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e049f5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.335597:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e049f5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.335604:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e049f5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760718.335718:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493672576:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44004us (44250us total) trans 4294972724 rc 0/0 00000100:00100000:1.0:1789760718.335733:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5435 00000100:00100000:1.0:1789760718.335743:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728574638856 time 728632727235 state 0x2 ref 0 00000100:00100000:3.0:1789760718.339054:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760718.339272:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493672704 00000100:00100000:1.0:1789760718.339282:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493672704 00000100:00100000:1.0:1789760718.339340:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728574638856 time 728632727235 state 0x0 ref 1 00000100:00100000:1.0:1789760718.339355:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728632987651 time 728632727235 state 0x1 ref 1 00000100:00100000:1.0:1789760718.339387:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728632987651 time 728680422334 state 0x0 ref 1 00000100:00100000:1.0:1789760718.339400:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493672704:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760718.339449:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22142976, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:1.0:1789760718.339458:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e049fc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.339473:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760718.339478:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.339483:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e049fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.339497:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e049fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760718.374261:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972725 is committed 00002000:00010000:3.0:1789760718.376027:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22142976 -> 22147072 00010000:00010000:3.0:1789760718.376048:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e049fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.376061:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e049fc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.376070:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e049fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.376078:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760718.376094:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e049fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.376104:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e049fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.376124:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e049fc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.376271:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e049fc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.376383:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493672704:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36979us (37332us total) trans 4294972725 rc 0/0 00000100:00100000:3.0:1789760718.376394:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5436 00000100:00100000:3.0:1789760718.376404:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728632987651 time 728680422334 state 0x2 ref 0 00000100:00100000:2.0:1789760718.383781:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760718.383923:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493672832 00000100:00100000:3.0:1789760718.383933:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493672832 00000100:00100000:3.0:1789760718.384029:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728632987651 time 728680422334 state 0x0 ref 1 00000100:00100000:3.0:1789760718.384048:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728680682750 time 728680422334 state 0x1 ref 1 00000100:00100000:3.0:1789760718.384080:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728680682750 time 728725115365 state 0x0 ref 1 00000100:00100000:3.0:1789760718.384092:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493672832:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760718.384157:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22147072, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:3.0:1789760718.384167:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04a03 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.384183:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760718.384189:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.384195:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04a03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.384209:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04a03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.394245:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760718.421569:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972726 is committed 00002000:00010000:2.0:1789760718.422325:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22147072 -> 22151168 00010000:00010000:2.0:1789760718.422340:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04a03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.422353:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04a03 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.422362:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04a03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.422371:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760718.422376:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04a03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.422386:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04a03 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.422405:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04a03 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.422413:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04a03 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.422717:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493672832:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38614us (38934us total) trans 4294972726 rc 0/0 00000100:00100000:2.0:1789760718.422728:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5437 00000100:00100000:2.0:1789760718.422738:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728680682750 time 728725115365 state 0x2 ref 0 00000100:00100000:1.0:1789760718.426490:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760718.426567:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493672960 00000100:00100000:2.0:1789760718.426576:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493672960 00000100:00100000:2.0:1789760718.426630:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728680682750 time 728725115365 state 0x0 ref 1 00000100:00100000:2.0:1789760718.426645:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728725375781 time 728725115365 state 0x1 ref 1 00000100:00100000:2.0:1789760718.426684:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728725375781 time 728767719490 state 0x0 ref 1 00000100:00100000:2.0:1789760718.426701:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc352c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493672960:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760718.426763:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22151168, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:2.0:1789760718.426773:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04a0a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.426791:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760718.426795:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760718.426801:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04a0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.426815:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04a0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760718.485621:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972727 is committed 00002000:00010000:3.0:1789760718.485795:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22151168 -> 22155264 00010000:00010000:3.0:1789760718.485811:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04a0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.485825:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04a0a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.485835:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04a0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.485843:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760718.485848:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04a0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.485858:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04a0a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.485890:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04a0a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.485899:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04a0a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.486038:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493672960:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59334us (59554us total) trans 4294972727 rc 0/0 00000100:00100000:3.0:1789760718.486049:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5438 00000100:00100000:3.0:1789760718.486060:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728725375781 time 728767719490 state 0x2 ref 0 00000100:00100000:1.0:1789760718.490843:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760718.490947:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493673088 00000100:00100000:2.0:1789760718.490958:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493673088 00000100:00100000:2.0:1789760718.491112:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728725375781 time 728767719490 state 0x0 ref 1 00000100:00100000:2.0:1789760718.491128:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728767979906 time 728767719490 state 0x1 ref 1 00000100:00100000:2.0:1789760718.491264:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728767979906 time 728832298457 state 0x0 ref 1 00000100:00100000:2.0:1789760718.491279:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc352ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493673088:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760718.491327:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22155264, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:2.0:1789760718.491337:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04a11 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.491354:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760718.491359:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760718.491365:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04a11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.491378:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04a11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760718.542157:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22155264 -> 22159360 00010000:00010000:3.0:1789760718.542178:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04a11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.542190:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04a11 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.542199:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04a11 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.542207:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760718.542212:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04a11 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.542221:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04a11 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.542241:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04a11 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.542249:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04a11 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.542351:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493673088:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51069us (51513us total) trans 4294972728 rc 0/0 00000100:00100000:3.0:1789760718.542361:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5439 00000100:00100000:3.0:1789760718.542370:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728767979906 time 728832298457 state 0x2 ref 0 00000001:00080000:0.0:1789760718.545762:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972728 is committed 00000100:00100000:2.0:1789760718.546430:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760718.546523:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493673216 00000100:00100000:3.0:1789760718.546532:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493673216 00000100:00100000:3.0:1789760718.546589:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728767979906 time 728832298457 state 0x0 ref 1 00000100:00100000:3.0:1789760718.546605:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728832558873 time 728832298457 state 0x1 ref 1 00000100:00100000:3.0:1789760718.546679:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728832558873 time 728887714800 state 0x0 ref 1 00000100:00100000:3.0:1789760718.546692:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a34e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493673216:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760718.546755:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22159360, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:3.0:1789760718.546764:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a18 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.546780:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760718.546785:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.546791:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.546804:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760718.590345:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972729 is committed 00002000:00010000:3.0:1789760718.590629:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22159360 -> 22163456 00010000:00010000:3.0:1789760718.590646:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.590671:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a18 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.590681:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.590688:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760718.590694:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.590765:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a18 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.590789:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a18 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.590797:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a18 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.591298:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a34e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493673216:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44602us (44868us total) trans 4294972729 rc 0/0 00000100:00100000:3.0:1789760718.591313:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5440 00000100:00100000:3.0:1789760718.591322:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728832558873 time 728887714800 state 0x2 ref 0 00000100:00100000:2.0:1789760718.595305:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760718.595408:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493673344 00000100:00100000:3.0:1789760718.595418:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493673344 00000100:00100000:3.0:1789760718.595472:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728832558873 time 728887714800 state 0x0 ref 1 00000100:00100000:3.0:1789760718.595487:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728887975216 time 728887714800 state 0x1 ref 1 00000100:00100000:3.0:1789760718.595557:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728887975216 time 728936592290 state 0x0 ref 1 00000100:00100000:3.0:1789760718.595571:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb05850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493673344:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760718.595624:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22163456, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0xb007 00010000:00010000:3.0:1789760718.595633:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a1f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.595658:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760718.595663:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.595670:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.595685:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760718.632487:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22163456 -> 22167552 00002000:00010000:3.0:1789760718.632496:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760717 -> 1789760718 00002000:00010000:3.0:1789760718.632499:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760717 -> 1789760718 00010000:00010000:3.0:1789760718.632512:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.632525:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a1f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.632534:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a1f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.632555:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760718.632560:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a1f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.632570:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a1f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.632588:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a1f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.632618:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a1f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.637758:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb05850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493673344:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42176us (42449us total) trans 4294972730 rc 0/0 00000100:00100000:3.0:1789760718.637775:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5441 00000100:00100000:3.0:1789760718.637788:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728887975216 time 728936592290 state 0x2 ref 0 00000001:00080000:0.0:1789760718.638250:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972730 is committed 00000100:00100000:1.0:1789760718.638789:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760718.638923:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493673472 00000100:00100000:2.0:1789760718.638932:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493673472 00000100:00100000:2.0:1789760718.639013:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728887975216 time 728936592290 state 0x0 ref 1 00000100:00100000:2.0:1789760718.639028:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728936852706 time 728936592290 state 0x1 ref 1 00000100:00100000:2.0:1789760718.639051:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728936852706 time 728980087300 state 0x0 ref 1 00000100:00100000:2.0:1789760718.639063:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc352c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493673472:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760718.639121:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22167552, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:2.0:1789760718.639139:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04a26 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.639154:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760718.639159:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760718.639165:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04a26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.639179:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04a26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760718.697148:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22167552 -> 22171648 00010000:00010000:2.0:1789760718.697171:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04a26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.697184:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04a26 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.697193:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04a26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.697200:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760718.697205:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04a26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.697215:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04a26 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.697236:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04a26 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.697244:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04a26 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.697347:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493673472:12345-192.168.203.60@tcp:4:dd.0 Request processed in 58279us (58553us total) trans 4294972731 rc 0/0 00000100:00100000:2.0:1789760718.697359:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5442 00000100:00100000:2.0:1789760718.697369:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728936852706 time 728980087300 state 0x2 ref 0 00000001:00080000:3.0:1789760718.698671:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972731 is committed 00000100:00100000:2.0:1789760718.700849:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760718.701198:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493673600 00000100:00100000:3.0:1789760718.701206:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493673600 00000100:00100000:3.0:1789760718.701259:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728936852706 time 728980087300 state 0x0 ref 1 00000100:00100000:3.0:1789760718.701469:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728980347716 time 728980087300 state 0x1 ref 1 00000100:00100000:3.0:1789760718.701533:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728980347716 time 729042568345 state 0x0 ref 1 00000100:00100000:3.0:1789760718.701548:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493673600:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760718.701595:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22171648, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:3.0:1789760718.701605:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a2d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.701621:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760718.701625:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.701632:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.701646:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.714103:0:3306:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789760718.714125:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff962101a35f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519249024:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760718.714204:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760718.714271:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519249024 00000100:00100000:2.0:1789760718.714278:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519249024 00000100:00100000:2.0:1789760718.714327:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 191 00000100:00100000:2.0:1789760718.714336:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a34e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519249024:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760718.714369:0:6690:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760718.714416:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a34e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519249024:12345-0@lo:13:osp-pre-1-0.0 Request processed in 80us (211us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760718.714423:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 191 00000100:00100000:2.0:1789760718.714470:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff962101a35f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519249024:0@lo:13:osp-pre-1-0.0 00000001:00080000:2.0:1789760718.740141:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972732 is committed 00002000:00010000:3.0:1789760718.740308:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22171648 -> 22175744 00010000:00010000:3.0:1789760718.740322:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.740339:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a2d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.740349:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.740356:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760718.740360:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.740369:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.740388:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a2d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.740395:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a2d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.740845:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493673600:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39294us (39999us total) trans 4294972732 rc 0/0 00000100:00100000:3.0:1789760718.740856:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5443 00000100:00100000:3.0:1789760718.740877:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728980347716 time 729042568345 state 0x2 ref 0 00000100:00100000:1.0:1789760718.747346:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760718.747426:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493673728 00000100:00100000:3.0:1789760718.747436:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493673728 00000100:00100000:3.0:1789760718.747487:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 728980347716 time 729042568345 state 0x0 ref 1 00000100:00100000:3.0:1789760718.747502:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729042828761 time 729042568345 state 0x1 ref 1 00000100:00100000:3.0:1789760718.747535:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729042828761 time 729088569910 state 0x0 ref 1 00000100:00100000:3.0:1789760718.747552:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc352ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493673728:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760718.747599:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22175744, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:3.0:1789760718.747607:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a34 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.747622:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760718.747627:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.747633:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.747660:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760718.783062:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972733 is committed 00002000:00010000:3.0:1789760718.783161:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22175744 -> 22179840 00010000:00010000:3.0:1789760718.783177:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.783190:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a34 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.783205:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.783213:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760718.783218:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.783228:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a34 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.783247:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a34 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.783254:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a34 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.783346:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493673728:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35791us (36005us total) trans 4294972733 rc 0/0 00000100:00100000:3.0:1789760718.783356:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5444 00000100:00100000:3.0:1789760718.783365:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729042828761 time 729088569910 state 0x2 ref 0 00000100:00100000:1.0:1789760718.787255:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760718.787351:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493673856 00000100:00100000:2.0:1789760718.787362:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493673856 00000100:00100000:2.0:1789760718.787590:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729042828761 time 729088569910 state 0x0 ref 1 00000100:00100000:2.0:1789760718.787608:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729088830326 time 729088569910 state 0x1 ref 1 00000100:00100000:2.0:1789760718.787790:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729088830326 time 729128825020 state 0x0 ref 1 00000100:00100000:2.0:1789760718.787803:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493673856:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760718.787850:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22179840, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:2.0:1789760718.787859:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04a3b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.787874:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760718.787879:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760718.787885:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04a3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.787899:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04a3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760718.820045:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972734 is committed 00002000:00010000:3.0:1789760718.820397:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22179840 -> 22183936 00010000:00010000:3.0:1789760718.820412:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04a3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.820431:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04a3b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.820441:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04a3b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.820516:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760718.820521:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04a3b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.820531:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04a3b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.820551:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04a3b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.820557:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04a3b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760718.820742:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493673856:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32935us (33491us total) trans 4294972734 rc 0/0 00000100:00100000:3.0:1789760718.820751:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5445 00000100:00100000:3.0:1789760718.820760:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729088830326 time 729128825020 state 0x2 ref 0 00000100:00100000:2.0:1789760718.825743:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760718.825859:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493673984 00000100:00100000:3.0:1789760718.825868:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493673984 00000100:00100000:3.0:1789760718.825919:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729088830326 time 729128825020 state 0x0 ref 1 00000100:00100000:3.0:1789760718.826097:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729129085436 time 729128825020 state 0x1 ref 1 00000100:00100000:3.0:1789760718.826124:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729129085436 time 729167159206 state 0x0 ref 1 00000100:00100000:3.0:1789760718.826137:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493673984:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760718.826189:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22183936, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:3.0:1789760718.826198:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a42 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.826214:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760718.826218:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.826225:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.826238:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760718.884837:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972735 is committed 00002000:00010000:2.0:1789760718.884949:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22183936 -> 22188032 00010000:00010000:2.0:1789760718.885074:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.885088:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a42 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.885097:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.885111:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760718.885116:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.885226:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a42 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.885375:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a42 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.889112:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04a42 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.889240:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493673984:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63101us (63502us total) trans 4294972735 rc 0/0 00000100:00100000:2.0:1789760718.889493:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5446 00000100:00100000:2.0:1789760718.889504:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729129085436 time 729167159206 state 0x2 ref 0 00000100:00100000:1.0:1789760718.901795:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760718.904263:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493674112 00000100:00100000:3.0:1789760718.904277:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493674112 00000100:00100000:3.0:1789760718.904333:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729129085436 time 729167159206 state 0x0 ref 1 00000100:00100000:3.0:1789760718.904348:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729167419622 time 729167159206 state 0x1 ref 1 00000100:00100000:3.0:1789760718.904387:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729167419622 time 729245423181 state 0x0 ref 1 00000100:00100000:3.0:1789760718.904403:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493674112:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760718.904449:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22188032, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:3.0:1789760718.904458:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a49 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.904472:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760718.904477:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760718.904483:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760718.904496:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760719.002231:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972736 is committed 00002000:00010000:0.0:1789760719.002443:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22188032 -> 22192128 00010000:00010000:0.0:1789760719.002460:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.002473:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a49 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.002481:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.002488:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760719.002493:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.002501:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a49 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.002520:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a49 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.002532:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04a49 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760719.002619:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493674112:12345-192.168.203.60@tcp:4:dd.0 Request processed in 98213us (100830us total) trans 4294972736 rc 0/0 00000100:00100000:0.0:1789760719.002629:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5447 00000100:00100000:0.0:1789760719.002637:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729167419622 time 729245423181 state 0x2 ref 0 00000100:00100000:2.0:1789760719.006905:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760719.007067:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493674240 00000100:00100000:0.0:1789760719.007077:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493674240 00000100:00100000:0.0:1789760719.007156:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729167419622 time 729245423181 state 0x0 ref 1 00000100:00100000:0.0:1789760719.007171:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729245683597 time 729245423181 state 0x1 ref 1 00000100:00100000:0.0:1789760719.007203:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729245683597 time 729348238662 state 0x0 ref 1 00000100:00100000:0.0:1789760719.007217:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493674240:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760719.007272:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22192128, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:0.0:1789760719.007281:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04a50 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.007298:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760719.007302:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.007308:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04a50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.007326:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04a50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760719.008163:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789760719.043492:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22192128 -> 22196224 00010000:00010000:1.0:1789760719.043513:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04a50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.043527:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04a50 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.043547:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04a50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.043554:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760719.043559:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04a50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.043569:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04a50 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.043587:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04a50 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.043596:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04a50 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.043709:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493674240:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36482us (36800us total) trans 4294972737 rc 0/0 00000100:00100000:1.0:1789760719.043720:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5448 00000100:00100000:1.0:1789760719.043729:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729245683597 time 729348238662 state 0x2 ref 0 00000001:00080000:2.0:1789760719.044548:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972737 is committed 00000100:00100000:2.0:1789760719.048423:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760719.048560:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493674368 00000100:00100000:3.0:1789760719.048570:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493674368 00000100:00100000:3.0:1789760719.048624:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729245683597 time 729348238662 state 0x0 ref 1 00000100:00100000:3.0:1789760719.048639:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729348499078 time 729348238662 state 0x1 ref 1 00000100:00100000:3.0:1789760719.048676:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729348499078 time 729389712173 state 0x0 ref 1 00000100:00100000:3.0:1789760719.048690:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7aa7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493674368:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760719.048749:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22196224, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:3.0:1789760719.048758:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04a57 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.048780:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760719.048784:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.048790:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04a57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.048803:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04a57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760719.097022:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22196224 -> 22200320 00010000:00010000:2.0:1789760719.097045:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04a57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.097063:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04a57 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.097072:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04a57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.097080:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760719.097086:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04a57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.097095:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04a57 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.097113:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04a57 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.097122:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04a57 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760719.097454:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7aa7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493674368:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48762us (49035us total) trans 4294972738 rc 0/0 00000100:00100000:2.0:1789760719.097465:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5449 00000100:00100000:2.0:1789760719.097474:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729348499078 time 729389712173 state 0x2 ref 0 00000001:00080000:0.0:1789760719.098483:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972738 is committed 00000100:00100000:0.0:1789760719.108051:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760719.108230:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493674496 00000100:00100000:3.0:1789760719.108240:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493674496 00000100:00100000:3.0:1789760719.108293:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729348499078 time 729389712173 state 0x0 ref 1 00000100:00100000:3.0:1789760719.108310:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729389972589 time 729389712173 state 0x1 ref 1 00000100:00100000:3.0:1789760719.108343:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729389972589 time 729449379001 state 0x0 ref 1 00000100:00100000:3.0:1789760719.108357:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493674496:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760719.108410:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22200320, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:3.0:1789760719.108420:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04a5e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.108435:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760719.108440:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.108446:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04a5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.108465:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04a5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760719.136507:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22200320 -> 22204416 00010000:00010000:2.0:1789760719.136528:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04a5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.136542:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04a5e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.136552:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04a5e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.136561:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760719.136565:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04a5e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.136575:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04a5e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.136593:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04a5e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.136601:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04a5e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760719.136704:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493674496:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28339us (28652us total) trans 4294972739 rc 0/0 00000100:00100000:2.0:1789760719.136715:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5450 00000100:00100000:2.0:1789760719.136723:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729389972589 time 729449379001 state 0x2 ref 0 00000001:00080000:0.0:1789760719.139709:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972739 is committed 00000100:00100000:1.0:1789760719.141054:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760719.141190:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493674624 00000100:00100000:0.0:1789760719.141200:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493674624 00000100:00100000:0.0:1789760719.141251:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729389972589 time 729449379001 state 0x0 ref 1 00000100:00100000:0.0:1789760719.141266:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729449639417 time 729449379001 state 0x1 ref 1 00000100:00100000:0.0:1789760719.141294:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729449639417 time 729482329924 state 0x0 ref 1 00000100:00100000:0.0:1789760719.141306:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493674624:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760719.141355:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22204416, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:0.0:1789760719.141364:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04a65 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.141379:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760719.141384:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.141389:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04a65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.141403:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04a65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760719.172393:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972740 is committed 00002000:00010000:3.0:1789760719.172507:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22204416 -> 22208512 00010000:00010000:3.0:1789760719.172522:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04a65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.172535:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04a65 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.172545:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04a65 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.172552:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760719.172557:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04a65 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.172567:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04a65 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.172584:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04a65 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.172592:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04a65 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760719.172687:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493674624:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31379us (31635us total) trans 4294972740 rc 0/0 00000100:00100000:3.0:1789760719.174243:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5451 00000100:00100000:3.0:1789760719.174261:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729449639417 time 729482329924 state 0x2 ref 0 00000100:00100000:2.0:1789760719.176294:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760719.176384:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493674752 00000100:00100000:0.0:1789760719.176394:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493674752 00000100:00100000:0.0:1789760719.176445:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729449639417 time 729482329924 state 0x0 ref 1 00000100:00100000:0.0:1789760719.176460:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729482590340 time 729482329924 state 0x1 ref 1 00000100:00100000:0.0:1789760719.176490:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729482590340 time 729517525582 state 0x0 ref 1 00000100:00100000:0.0:1789760719.176504:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8794700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493674752:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760719.176555:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22208512, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:0.0:1789760719.176565:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04a6c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.176580:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760719.176584:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.176590:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04a6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.176604:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04a6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760719.204994:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972741 is committed 00002000:00010000:2.0:1789760719.205911:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22208512 -> 22212608 00010000:00010000:2.0:1789760719.205928:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04a6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.205940:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04a6c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.206044:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04a6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.206053:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760719.206058:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04a6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.206069:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04a6c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.206089:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04a6c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.206096:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04a6c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760719.206461:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8794700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493674752:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29954us (30168us total) trans 4294972741 rc 0/0 00000100:00100000:2.0:1789760719.206475:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5452 00000100:00100000:2.0:1789760719.206484:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729482590340 time 729517525582 state 0x2 ref 0 00000100:00100000:0.0:1789760719.211298:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760719.211396:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493674880 00000100:00100000:3.0:1789760719.211407:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493674880 00000100:00100000:3.0:1789760719.211465:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729482590340 time 729517525582 state 0x0 ref 1 00000100:00100000:3.0:1789760719.211480:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729517785998 time 729517525582 state 0x1 ref 1 00000100:00100000:3.0:1789760719.211547:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729517785998 time 729552581856 state 0x0 ref 1 00000100:00100000:3.0:1789760719.211563:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493674880:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760719.211611:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22212608, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:3.0:1789760719.211619:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04a73 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.211635:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760719.211640:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.211654:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04a73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.211668:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04a73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760719.258674:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22212608 -> 22216704 00010000:00010000:3.0:1789760719.258693:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04a73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.258772:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04a73 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.258782:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04a73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.258789:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760719.258794:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04a73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.258804:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04a73 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.258824:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04a73 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.258831:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04a73 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760719.258924:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493674880:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47361us (47631us total) trans 4294972742 rc 0/0 00000100:00100000:3.0:1789760719.258938:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5453 00000100:00100000:3.0:1789760719.258947:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729517785998 time 729552581856 state 0x2 ref 0 00000001:00080000:3.0:1789760719.259013:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972742 is committed 00000100:00100000:0.0:1789760719.266077:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760719.266166:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493675008 00000100:00100000:2.0:1789760719.266176:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493675008 00000100:00100000:2.0:1789760719.266233:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729517785998 time 729552581856 state 0x0 ref 1 00000100:00100000:2.0:1789760719.266250:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729552842272 time 729552581856 state 0x1 ref 1 00000100:00100000:2.0:1789760719.266282:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729552842272 time 729607316759 state 0x0 ref 1 00000100:00100000:2.0:1789760719.266295:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493675008:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760719.266344:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22216704, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:2.0:1789760719.266352:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04a7a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.266368:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760719.266373:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760719.266378:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04a7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.266392:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04a7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760719.292214:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972743 is committed 00002000:00010000:1.0:1789760719.292359:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22216704 -> 22220800 00010000:00010000:1.0:1789760719.292373:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04a7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.292387:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04a7a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.292395:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04a7a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.292403:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760719.292408:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04a7a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.292417:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04a7a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.292437:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04a7a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.292451:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04a7a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.292543:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493675008:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26246us (26471us total) trans 4294972743 rc 0/0 00000100:00100000:1.0:1789760719.292554:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5454 00000100:00100000:1.0:1789760719.292563:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729552842272 time 729607316759 state 0x2 ref 0 00000100:00100000:2.0:1789760719.296506:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760719.296602:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493675136 00000100:00100000:0.0:1789760719.296612:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493675136 00000100:00100000:0.0:1789760719.296675:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729552842272 time 729607316759 state 0x0 ref 1 00000100:00100000:0.0:1789760719.296689:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729607577175 time 729607316759 state 0x1 ref 1 00000100:00100000:0.0:1789760719.296812:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729607577175 time 729637846065 state 0x0 ref 1 00000100:00100000:0.0:1789760719.296825:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8796a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493675136:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760719.296871:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22220800, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:0.0:1789760719.296880:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04a81 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.296895:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760719.296900:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.296907:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04a81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.296922:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04a81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760719.343814:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972744 is committed 00002000:00010000:3.0:1789760719.343979:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22220800 -> 22224896 00010000:00010000:3.0:1789760719.343993:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04a81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.344005:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04a81 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.344014:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04a81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.344020:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760719.344025:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04a81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.344033:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04a81 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.344051:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04a81 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.344058:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04a81 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760719.344279:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8796a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493675136:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47450us (47776us total) trans 4294972744 rc 0/0 00000100:00100000:3.0:1789760719.344290:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5455 00000100:00100000:3.0:1789760719.344300:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729607577175 time 729637846065 state 0x2 ref 0 00000100:00100000:2.0:1789760719.348095:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760719.348247:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493675264 00000100:00100000:0.0:1789760719.348257:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493675264 00000100:00100000:0.0:1789760719.348309:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729607577175 time 729637846065 state 0x0 ref 1 00000100:00100000:0.0:1789760719.348374:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729638106481 time 729637846065 state 0x1 ref 1 00000100:00100000:0.0:1789760719.348442:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729638106481 time 729689476161 state 0x0 ref 1 00000100:00100000:0.0:1789760719.348458:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493675264:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760719.348509:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22224896, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:0.0:1789760719.348518:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e04a88 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.348535:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760719.348539:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.348545:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e04a88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.348559:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e04a88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760719.394707:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972745 is committed 00002000:00010000:0.0:1789760719.395024:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22224896 -> 22228992 00010000:00010000:0.0:1789760719.395039:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e04a88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.395051:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e04a88 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.395059:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e04a88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.395065:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760719.395069:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e04a88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.395078:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e04a88 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.395097:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e04a88 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.395106:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e04a88 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760719.395294:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493675264:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46834us (47202us total) trans 4294972745 rc 0/0 00000100:00100000:0.0:1789760719.395305:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5456 00000100:00100000:0.0:1789760719.395314:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729638106481 time 729689476161 state 0x2 ref 0 00000100:00100000:2.0:1789760719.400788:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760719.401011:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493675392 00000100:00100000:3.0:1789760719.401224:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493675392 00000100:00100000:2.0:1789760719.401341:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729638106481 time 729689476161 state 0x0 ref 1 00000100:00100000:2.0:1789760719.401358:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729689736577 time 729689476161 state 0x1 ref 1 00000100:00100000:2.0:1789760719.401391:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729689736577 time 729742426560 state 0x0 ref 1 00000100:00100000:2.0:1789760719.401404:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493675392:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760719.401621:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22228992, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:2.0:1789760719.401630:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04a8f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.401654:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760719.401659:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760719.401664:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04a8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.401678:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04a8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.418203:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760719.474854:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22228992 -> 22233088 00010000:00010000:1.0:1789760719.474883:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04a8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.474897:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04a8f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.474905:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04a8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.474913:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760719.474918:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04a8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.474927:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04a8f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.474951:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04a8f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.475002:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04a8f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.475111:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493675392:12345-192.168.203.60@tcp:4:dd.0 Request processed in 73696us (74321us total) trans 4294972746 rc 0/0 00000100:00100000:1.0:1789760719.475123:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5457 00000100:00100000:1.0:1789760719.475132:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729689736577 time 729742426560 state 0x2 ref 0 00000001:00080000:3.0:1789760719.476523:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972746 is committed 00000100:00100000:2.0:1789760719.481594:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760719.481770:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493675520 00000100:00100000:1.0:1789760719.481779:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493675520 00000100:00100000:1.0:1789760719.481829:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729689736577 time 729742426560 state 0x0 ref 1 00000100:00100000:1.0:1789760719.481841:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729742686976 time 729742426560 state 0x1 ref 1 00000100:00100000:1.0:1789760719.481915:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729742686976 time 729822950216 state 0x0 ref 1 00000100:00100000:1.0:1789760719.481927:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493675520:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760719.482007:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22233088, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:1.0:1789760719.482015:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04a96 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.482037:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760719.482042:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760719.482048:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04a96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.482061:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04a96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760719.537277:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972747 is committed 00002000:00010000:2.0:1789760719.537398:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22233088 -> 22237184 00010000:00010000:2.0:1789760719.537414:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04a96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.537428:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04a96 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.537437:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04a96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.537446:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760719.537450:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04a96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.537460:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04a96 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.537478:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04a96 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.537487:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04a96 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760719.537624:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493675520:12345-192.168.203.60@tcp:4:dd.0 Request processed in 55693us (56035us total) trans 4294972747 rc 0/0 00000100:00100000:2.0:1789760719.537635:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5458 00000100:00100000:2.0:1789760719.537655:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729742686976 time 729822950216 state 0x2 ref 0 00000100:00100000:0.0:1789760719.541539:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760719.541627:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493675648 00000100:00100000:3.0:1789760719.541636:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493675648 00000100:00100000:3.0:1789760719.541695:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729742686976 time 729822950216 state 0x0 ref 1 00000100:00100000:3.0:1789760719.541766:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729823210632 time 729822950216 state 0x1 ref 1 00000100:00100000:3.0:1789760719.541829:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729823210632 time 729882864409 state 0x0 ref 1 00000100:00100000:3.0:1789760719.541842:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493675648:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760719.541890:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22237184, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:3.0:1789760719.541899:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04a9d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.541915:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760719.541928:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.541934:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04a9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.541948:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04a9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760719.589002:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22237184 -> 22241280 00010000:00010000:0.0:1789760719.589022:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04a9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.589036:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04a9d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.589044:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04a9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.589057:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760719.589062:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04a9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.589071:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04a9d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.589092:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04a9d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.589099:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04a9d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760719.589207:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493675648:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47361us (47672us total) trans 4294972748 rc 0/0 00000100:00100000:0.0:1789760719.589217:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5459 00000100:00100000:0.0:1789760719.589227:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729823210632 time 729882864409 state 0x2 ref 0 00000100:00100000:0.0:1789760719.592605:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760719.592704:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493675776 00000100:00100000:3.0:1789760719.592713:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493675776 00000100:00100000:3.0:1789760719.592767:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729823210632 time 729882864409 state 0x0 ref 1 00000100:00100000:3.0:1789760719.592781:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729883124825 time 729882864409 state 0x1 ref 1 00000100:00100000:3.0:1789760719.592813:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729883124825 time 729933848981 state 0x0 ref 1 00000100:00100000:3.0:1789760719.592825:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493675776:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760719.592873:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22241280, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0xb007 00010000:00010000:3.0:1789760719.592881:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04aa4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.592896:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760719.592900:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760719.592906:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04aa4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760719.592920:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04aa4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760719.596222:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972748 is committed 00002000:00010000:2.0:1789760719.619042:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22241280 -> 22245376 00002000:00010000:2.0:1789760719.619051:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760718 -> 1789760719 00002000:00010000:2.0:1789760719.619054:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760718 -> 1789760719 00010000:00010000:2.0:1789760719.619068:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04aa4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.619082:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04aa4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.619091:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04aa4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.619099:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760719.619103:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04aa4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.619112:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04aa4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.619199:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04aa4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.619208:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04aa4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760719.619297:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493675776:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26469us (26697us total) trans 4294972749 rc 0/0 00000100:00100000:2.0:1789760719.619308:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5460 00000100:00100000:2.0:1789760719.619317:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729883124825 time 729933848981 state 0x2 ref 0 00000001:00080000:1.0:1789760719.622621:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972749 is committed 00000100:00100000:2.0:1789760719.623460:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760719.623551:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493675904 00000100:00100000:1.0:1789760719.623561:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493675904 00000100:00100000:1.0:1789760719.623613:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729883124825 time 729933848981 state 0x0 ref 1 00000100:00100000:1.0:1789760719.623628:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729934109397 time 729933848981 state 0x1 ref 1 00000100:00100000:1.0:1789760719.623709:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729934109397 time 729964743803 state 0x0 ref 1 00000100:00100000:1.0:1789760719.623720:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493675904:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760719.623767:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22245376, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:1.0:1789760719.623775:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04aab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.623790:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760719.623795:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760719.623800:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04aab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.623812:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04aab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760719.684553:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972750 is committed 00002000:00010000:1.0:1789760719.685145:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22245376 -> 22249472 00010000:00010000:1.0:1789760719.685161:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04aab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.685173:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04aab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.685183:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04aab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.685191:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760719.685206:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04aab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.685216:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04aab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.685402:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04aab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.685413:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04aab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.685541:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493675904:12345-192.168.203.60@tcp:4:dd.0 Request processed in 61818us (62082us total) trans 4294972750 rc 0/0 00000100:00100000:1.0:1789760719.685552:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5461 00000100:00100000:1.0:1789760719.685560:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729934109397 time 729964743803 state 0x2 ref 0 00000100:00100000:0.0:1789760719.696327:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760719.696532:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493676032 00000100:00100000:2.0:1789760719.696543:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493676032 00000100:00100000:2.0:1789760719.696599:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729934109397 time 729964743803 state 0x0 ref 1 00000100:00100000:2.0:1789760719.696614:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729965004219 time 729964743803 state 0x1 ref 1 00000100:00100000:2.0:1789760719.696709:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729965004219 time 730037743461 state 0x0 ref 1 00000100:00100000:2.0:1789760719.696724:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493676032:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760719.696774:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22249472, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:2.0:1789760719.696783:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04ab2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.696798:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760719.696802:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760719.696808:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04ab2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.696823:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04ab2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760719.771405:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22249472 -> 22253568 00010000:00010000:0.0:1789760719.771425:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04ab2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.771527:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04ab2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.771539:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04ab2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.771546:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760719.771550:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04ab2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.771559:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04ab2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.771577:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04ab2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.771584:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04ab2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760719.771677:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493676032:12345-192.168.203.60@tcp:4:dd.0 Request processed in 74951us (75358us total) trans 4294972751 rc 0/0 00000100:00100000:0.0:1789760719.771687:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5462 00000100:00100000:0.0:1789760719.771760:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729965004219 time 730037743461 state 0x2 ref 0 00000001:00080000:3.0:1789760719.775720:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972751 is committed 00000100:00100000:1.0:1789760719.778497:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760719.778628:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493676160 00000100:00100000:0.0:1789760719.778638:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493676160 00000100:00100000:0.0:1789760719.778712:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 729965004219 time 730037743461 state 0x0 ref 1 00000100:00100000:0.0:1789760719.778729:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730038003877 time 730037743461 state 0x1 ref 1 00000100:00100000:0.0:1789760719.778757:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730038003877 time 730119793026 state 0x0 ref 1 00000100:00100000:0.0:1789760719.778770:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9621018a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493676160:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760719.778820:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22253568, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:0.0:1789760719.778829:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04ab9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.778845:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760719.778850:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.778867:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04ab9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.778881:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04ab9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760719.839161:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972752 is committed 00002000:00010000:1.0:1789760719.839362:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22253568 -> 22257664 00010000:00010000:1.0:1789760719.839378:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04ab9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.839390:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04ab9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.839399:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04ab9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.839406:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760719.839410:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04ab9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.839419:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04ab9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.839449:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04ab9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.839458:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04ab9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.839562:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9621018a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493676160:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60789us (61070us total) trans 4294972752 rc 0/0 00000100:00100000:1.0:1789760719.839573:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5463 00000100:00100000:1.0:1789760719.839582:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730038003877 time 730119793026 state 0x2 ref 0 00000100:00100000:2.0:1789760719.845900:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760719.846210:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493676288 00000100:00100000:1.0:1789760719.846221:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493676288 00000100:00100000:1.0:1789760719.846274:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730038003877 time 730119793026 state 0x0 ref 1 00000100:00100000:1.0:1789760719.846291:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730120053442 time 730119793026 state 0x1 ref 1 00000100:00100000:1.0:1789760719.846352:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730120053442 time 730187386303 state 0x0 ref 1 00000100:00100000:1.0:1789760719.846366:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493676288:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760719.846416:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22257664, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:1.0:1789760719.846425:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04ac0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.846505:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760719.846511:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760719.846517:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04ac0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.846531:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04ac0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760719.904347:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972753 is committed 00002000:00010000:1.0:1789760719.904432:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22257664 -> 22261760 00010000:00010000:1.0:1789760719.904519:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04ac0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.904532:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04ac0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.904541:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04ac0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.904548:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760719.904553:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04ac0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.904561:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04ac0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.904580:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04ac0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.904587:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04ac0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.904789:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493676288:12345-192.168.203.60@tcp:4:dd.0 Request processed in 58421us (58891us total) trans 4294972753 rc 0/0 00000100:00100000:1.0:1789760719.904804:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5464 00000100:00100000:1.0:1789760719.904813:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730120053442 time 730187386303 state 0x2 ref 0 00000100:00100000:2.0:1789760719.912983:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760719.913314:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493676416 00000100:00100000:1.0:1789760719.913324:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493676416 00000100:00100000:1.0:1789760719.913381:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730120053442 time 730187386303 state 0x0 ref 1 00000100:00100000:1.0:1789760719.913396:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730187646719 time 730187386303 state 0x1 ref 1 00000100:00100000:1.0:1789760719.913430:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730187646719 time 730254465831 state 0x0 ref 1 00000100:00100000:1.0:1789760719.913554:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493676416:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760719.913601:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22261760, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:1.0:1789760719.913609:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04ac7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.913623:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760719.913627:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760719.913633:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04ac7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.913658:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04ac7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760719.959527:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972754 is committed 00002000:00010000:2.0:1789760719.959759:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22261760 -> 22265856 00010000:00010000:2.0:1789760719.959774:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04ac7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.959794:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04ac7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.959803:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04ac7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.959810:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760719.959815:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04ac7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.959823:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04ac7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.959841:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04ac7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.959848:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04ac7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760719.959950:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493676416:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46396us (46997us total) trans 4294972754 rc 0/0 00000100:00100000:2.0:1789760719.959994:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5465 00000100:00100000:2.0:1789760719.960004:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730187646719 time 730254465831 state 0x2 ref 0 00000100:00100000:2.0:1789760719.965562:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760719.965854:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493676544 00000100:00100000:0.0:1789760719.965873:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493676544 00000100:00100000:0.0:1789760719.965923:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730187646719 time 730254465831 state 0x0 ref 1 00000100:00100000:0.0:1789760719.965937:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730254726247 time 730254465831 state 0x1 ref 1 00000100:00100000:0.0:1789760719.965990:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730254726247 time 730307025696 state 0x0 ref 1 00000100:00100000:0.0:1789760719.966002:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493676544:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760719.966050:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22265856, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:0.0:1789760719.966059:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04ace lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.966310:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760719.966315:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.966322:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04ace lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.966337:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04ace lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760720.015855:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972755 is committed 00002000:00010000:2.0:1789760720.016031:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22265856 -> 22269952 00010000:00010000:2.0:1789760720.016046:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04ace lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.016199:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04ace lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.016208:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04ace lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.016215:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760720.016219:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04ace lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.016228:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04ace lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.016247:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04ace lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.016254:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04ace lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760720.016393:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493676544:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50387us (50835us total) trans 4294972755 rc 0/0 00000100:00100000:2.0:1789760720.016404:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5466 00000100:00100000:2.0:1789760720.016414:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730254726247 time 730307025696 state 0x2 ref 0 00000100:00100000:0.0:1789760720.021754:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760720.021883:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493676672 00000100:00100000:2.0:1789760720.021893:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493676672 00000100:00100000:2.0:1789760720.021957:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730254726247 time 730307025696 state 0x0 ref 1 00000100:00100000:2.0:1789760720.022012:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730307286112 time 730307025696 state 0x1 ref 1 00000100:00100000:2.0:1789760720.022044:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730307286112 time 730363079024 state 0x0 ref 1 00000100:00100000:2.0:1789760720.022059:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493676672:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760720.022121:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22269952, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:2.0:1789760720.022130:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04ad5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.022146:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760720.022151:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760720.022157:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04ad5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.022170:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04ad5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760720.066426:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22269952 -> 22274048 00010000:00010000:1.0:1789760720.066445:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04ad5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.066459:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04ad5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.066469:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04ad5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.066477:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760720.066481:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04ad5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.066490:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04ad5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.066510:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04ad5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.066518:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04ad5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.066608:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493676672:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44548us (44860us total) trans 4294972756 rc 0/0 00000100:00100000:1.0:1789760720.066619:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5467 00000100:00100000:1.0:1789760720.066629:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730307286112 time 730363079024 state 0x2 ref 0 00000001:00080000:0.0:1789760720.066805:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972756 is committed 00000100:00100000:0.0:1789760720.070630:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760720.070717:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493676800 00000100:00100000:1.0:1789760720.070726:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493676800 00000100:00100000:1.0:1789760720.070783:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730307286112 time 730363079024 state 0x0 ref 1 00000100:00100000:1.0:1789760720.070798:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730363339440 time 730363079024 state 0x1 ref 1 00000100:00100000:1.0:1789760720.070831:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730363339440 time 730411866437 state 0x0 ref 1 00000100:00100000:1.0:1789760720.070845:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493676800:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760720.070907:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22274048, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:1.0:1789760720.070915:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04adc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.070930:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760720.070935:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.070942:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04adc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.070956:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04adc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760720.123950:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22274048 -> 22278144 00010000:00010000:1.0:1789760720.123994:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04adc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.124007:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04adc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.124023:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04adc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.124030:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760720.124035:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04adc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.124044:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04adc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.124064:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04adc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760720.124269:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972757 is committed 00010000:00010000:1.0:1789760720.124316:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04adc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760720.124500:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493676800:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53651us (53873us total) trans 4294972757 rc 0/0 00000100:00100000:2.0:1789760720.124512:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5468 00000100:00100000:2.0:1789760720.124521:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730363339440 time 730411866437 state 0x2 ref 0 00000100:00100000:2.0:1789760720.130033:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760720.130256:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493676928 00000100:00100000:1.0:1789760720.130267:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493676928 00000100:00100000:1.0:1789760720.130326:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730363339440 time 730411866437 state 0x0 ref 1 00000100:00100000:1.0:1789760720.130341:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730412126853 time 730411866437 state 0x1 ref 1 00000100:00100000:1.0:1789760720.130409:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730412126853 time 730471443654 state 0x0 ref 1 00000100:00100000:1.0:1789760720.130422:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493676928:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760720.130478:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22278144, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:1.0:1789760720.130487:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04ae3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.130504:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760720.130509:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.130515:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04ae3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.130530:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04ae3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760720.212585:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972758 is committed 00002000:00010000:2.0:1789760720.212818:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22278144 -> 22282240 00010000:00010000:2.0:1789760720.212836:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04ae3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.212850:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04ae3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.212932:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04ae3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.212941:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760720.212946:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04ae3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.212956:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04ae3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.213044:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04ae3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.213053:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04ae3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760720.213414:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493676928:12345-192.168.203.60@tcp:4:dd.0 Request processed in 82989us (83452us total) trans 4294972758 rc 0/0 00000100:00100000:2.0:1789760720.213430:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5469 00000100:00100000:2.0:1789760720.213440:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730412126853 time 730471443654 state 0x2 ref 0 00000100:00100000:0.0:1789760720.222490:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760720.222584:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493677056 00000100:00100000:2.0:1789760720.222594:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493677056 00000100:00100000:2.0:1789760720.222659:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730412126853 time 730471443654 state 0x0 ref 1 00000100:00100000:2.0:1789760720.222676:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730471704070 time 730471443654 state 0x1 ref 1 00000100:00100000:2.0:1789760720.222715:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730471704070 time 730563750357 state 0x0 ref 1 00000100:00100000:2.0:1789760720.222728:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493677056:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760720.222775:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22282240, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:2.0:1789760720.222784:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04aea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.222799:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760720.222804:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760720.222810:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04aea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.222825:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04aea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760720.309575:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972759 is committed 00002000:00010000:2.0:1789760720.311028:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22282240 -> 22286336 00010000:00010000:2.0:1789760720.311045:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04aea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.311061:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04aea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.311070:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04aea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.311079:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760720.311083:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04aea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.311093:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04aea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.311113:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04aea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.311122:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04aea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760720.311313:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493677056:12345-192.168.203.60@tcp:4:dd.0 Request processed in 88582us (88828us total) trans 4294972759 rc 0/0 00000100:00100000:2.0:1789760720.311325:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5470 00000100:00100000:2.0:1789760720.311335:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730471704070 time 730563750357 state 0x2 ref 0 00000100:00100000:2.0:1789760720.314736:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760720.314833:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493677184 00000100:00100000:1.0:1789760720.314842:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493677184 00000100:00100000:1.0:1789760720.314909:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730471704070 time 730563750357 state 0x0 ref 1 00000100:00100000:1.0:1789760720.314928:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730564010773 time 730563750357 state 0x1 ref 1 00000100:00100000:1.0:1789760720.314955:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730564010773 time 730655990446 state 0x0 ref 1 00000100:00100000:1.0:1789760720.314998:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493677184:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760720.315048:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22286336, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:1.0:1789760720.315057:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04af1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.315080:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760720.315085:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.315092:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04af1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.315106:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04af1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760720.382124:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22286336 -> 22290432 00010000:00010000:2.0:1789760720.382143:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04af1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.382156:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04af1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.382165:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04af1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.382173:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760720.382178:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04af1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.382188:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04af1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.382207:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04af1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.382214:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04af1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760720.382305:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493677184:12345-192.168.203.60@tcp:4:dd.0 Request processed in 67305us (67573us total) trans 4294972760 rc 0/0 00000100:00100000:2.0:1789760720.382316:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5471 00000100:00100000:2.0:1789760720.382325:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730564010773 time 730655990446 state 0x2 ref 0 00000001:00080000:1.0:1789760720.382776:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972760 is committed 00000100:00100000:1.0:1789760720.392902:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760720.398852:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493677312 00000100:00100000:0.0:1789760720.398868:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493677312 00000100:00100000:2.0:1789760720.403424:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730564010773 time 730655990446 state 0x0 ref 1 00000100:00100000:2.0:1789760720.403441:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730656250862 time 730655990446 state 0x1 ref 1 00000100:00100000:2.0:1789760720.403472:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730656250862 time 730744508345 state 0x0 ref 1 00000100:00100000:2.0:1789760720.403484:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9621018a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493677312:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760720.403532:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22290432, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:2.0:1789760720.403541:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04af8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.403556:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760720.403560:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760720.403567:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04af8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.403581:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04af8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.448038:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760720.470047:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972761 is committed 00002000:00010000:2.0:1789760720.470245:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22290432 -> 22294528 00010000:00010000:2.0:1789760720.470259:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04af8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.470273:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04af8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.470290:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04af8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.470298:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760720.470303:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04af8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.470312:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04af8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.470331:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04af8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.470341:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04af8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760720.470432:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9621018a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493677312:12345-192.168.203.60@tcp:4:dd.0 Request processed in 66944us (77535us total) trans 4294972761 rc 0/0 00000100:00100000:2.0:1789760720.470441:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5472 00000100:00100000:2.0:1789760720.470451:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730656250862 time 730744508345 state 0x2 ref 0 00000100:00100000:0.0:1789760720.473952:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760720.474108:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493677440 00000100:00100000:2.0:1789760720.474127:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493677440 00000100:00100000:2.0:1789760720.474178:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730656250862 time 730744508345 state 0x0 ref 1 00000100:00100000:2.0:1789760720.474192:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730744768761 time 730744508345 state 0x1 ref 1 00000100:00100000:2.0:1789760720.474222:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730744768761 time 730815258645 state 0x0 ref 1 00000100:00100000:2.0:1789760720.474234:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493677440:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760720.474280:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22294528, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:2.0:1789760720.474289:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04aff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.474305:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760720.474310:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760720.474315:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04aff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.474329:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04aff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760720.535183:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972762 is committed 00002000:00010000:3.0:1789760720.535264:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22294528 -> 22298624 00010000:00010000:3.0:1789760720.535279:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04aff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.535291:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04aff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.535299:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04aff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.535306:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760720.535311:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04aff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.535318:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04aff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.535338:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04aff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.535345:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04aff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760720.535436:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493677440:12345-192.168.203.60@tcp:4:dd.0 Request processed in 61199us (61489us total) trans 4294972762 rc 0/0 00000100:00100000:3.0:1789760720.535447:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5473 00000100:00100000:3.0:1789760720.535456:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730744768761 time 730815258645 state 0x2 ref 0 00000100:00100000:0.0:1789760720.543153:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760720.543309:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493677568 00000100:00100000:2.0:1789760720.543320:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493677568 00000100:00100000:2.0:1789760720.543380:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730744768761 time 730815258645 state 0x0 ref 1 00000100:00100000:2.0:1789760720.543395:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730815519061 time 730815258645 state 0x1 ref 1 00000100:00100000:2.0:1789760720.543427:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730815519061 time 730884463168 state 0x0 ref 1 00000100:00100000:2.0:1789760720.543440:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493677568:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760720.543499:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22298624, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:2.0:1789760720.543508:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04b06 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.543524:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760720.543528:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760720.543534:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04b06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.543548:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04b06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760720.615660:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972763 is committed 00002000:00010000:3.0:1789760720.615930:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22298624 -> 22302720 00010000:00010000:3.0:1789760720.615944:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04b06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.615958:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04b06 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.615987:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04b06 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.615994:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760720.615999:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04b06 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.616007:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04b06 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.616027:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04b06 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.616035:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04b06 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760720.616132:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493677568:12345-192.168.203.60@tcp:4:dd.0 Request processed in 72689us (72983us total) trans 4294972763 rc 0/0 00000100:00100000:3.0:1789760720.616142:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5474 00000100:00100000:3.0:1789760720.616152:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730815519061 time 730884463168 state 0x2 ref 0 00000100:00100000:2.0:1789760720.623125:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760720.623411:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493677696 00000100:00100000:1.0:1789760720.623420:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493677696 00000100:00100000:1.0:1789760720.623472:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730815519061 time 730884463168 state 0x0 ref 1 00000100:00100000:1.0:1789760720.623486:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730884723584 time 730884463168 state 0x1 ref 1 00000100:00100000:1.0:1789760720.623514:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730884723584 time 730964550507 state 0x0 ref 1 00000100:00100000:1.0:1789760720.623525:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493677696:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760720.623568:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22302720, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0xb007 00010000:00010000:1.0:1789760720.623577:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04b0d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.623592:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760720.623596:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.623604:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04b0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.623617:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04b0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760720.719380:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972764 is committed 00002000:00010000:2.0:1789760720.719812:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22302720 -> 22306816 00002000:00010000:2.0:1789760720.719818:0:13038:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760719 -> 1789760720 00002000:00010000:2.0:1789760720.719821:0:13038:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760719 -> 1789760720 00010000:00010000:2.0:1789760720.719833:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04b0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.719846:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04b0d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.719856:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04b0d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.719864:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760720.719871:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04b0d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.719882:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04b0d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.719903:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04b0d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.719911:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04b0d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760720.720595:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493677696:12345-192.168.203.60@tcp:4:dd.0 Request processed in 97063us (97482us total) trans 4294972764 rc 0/0 00000100:00100000:2.0:1789760720.720609:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5475 00000100:00100000:2.0:1789760720.720620:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730884723584 time 730964550507 state 0x2 ref 0 00000100:00100000:0.0:1789760720.728252:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760720.728344:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493677824 00000100:00100000:1.0:1789760720.728352:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493677824 00000100:00100000:1.0:1789760720.728402:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730884723584 time 730964550507 state 0x0 ref 1 00000100:00100000:1.0:1789760720.728417:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730964810923 time 730964550507 state 0x1 ref 1 00000100:00100000:1.0:1789760720.728447:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730964810923 time 731069482793 state 0x0 ref 1 00000100:00100000:1.0:1789760720.728459:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493677824:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760720.728508:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22306816, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:1.0:1789760720.728518:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b14 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.728533:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760720.728538:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.728544:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.728557:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760720.761657:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972765 is committed 00002000:00010000:1.0:1789760720.761796:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22306816 -> 22310912 00010000:00010000:1.0:1789760720.761812:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.761826:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b14 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.761834:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b14 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.761841:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760720.761846:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b14 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.761855:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b14 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.761874:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b14 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.761883:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b14 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.762051:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493677824:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33588us (33811us total) trans 4294972765 rc 0/0 00000100:00100000:1.0:1789760720.762065:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5476 00000100:00100000:1.0:1789760720.762075:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730964810923 time 731069482793 state 0x2 ref 0 00000100:00100000:0.0:1789760720.765437:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760720.765558:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493677952 00000100:00100000:3.0:1789760720.765569:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493677952 00000100:00100000:3.0:1789760720.765624:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 730964810923 time 731069482793 state 0x0 ref 1 00000100:00100000:3.0:1789760720.765651:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731069743209 time 731069482793 state 0x1 ref 1 00000100:00100000:3.0:1789760720.765682:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731069743209 time 731106717996 state 0x0 ref 1 00000100:00100000:3.0:1789760720.765704:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493677952:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760720.765752:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22310912, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:3.0:1789760720.765761:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b1b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.765776:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760720.765780:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760720.765786:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.765799:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760720.836499:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972766 is committed 00002000:00010000:0.0:1789760720.836587:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22310912 -> 22315008 00010000:00010000:0.0:1789760720.836602:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.836617:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b1b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.836627:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b1b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.836649:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760720.836654:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b1b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.836733:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b1b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.836754:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b1b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.836762:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b1b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760720.836852:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493677952:12345-192.168.203.60@tcp:4:dd.0 Request processed in 71147us (71421us total) trans 4294972766 rc 0/0 00000100:00100000:0.0:1789760720.836862:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5477 00000100:00100000:0.0:1789760720.836872:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731069743209 time 731106717996 state 0x2 ref 0 00000100:00100000:2.0:1789760720.842827:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760720.842926:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493678080 00000100:00100000:3.0:1789760720.842937:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493678080 00000100:00100000:3.0:1789760720.843238:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731069743209 time 731106717996 state 0x0 ref 1 00000100:00100000:3.0:1789760720.843253:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731106978412 time 731106717996 state 0x1 ref 1 00000100:00100000:2.0:1789760720.843372:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731106978412 time 731184405712 state 0x0 ref 1 00000100:00100000:2.0:1789760720.843389:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493678080:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760720.843520:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22315008, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:2.0:1789760720.843530:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04b22 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.843552:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760720.843556:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760720.843562:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04b22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.843576:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04b22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760720.916506:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972767 is committed 00002000:00010000:1.0:1789760720.916583:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22315008 -> 22319104 00010000:00010000:1.0:1789760720.916673:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04b22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.916750:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04b22 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.916760:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04b22 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.916768:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760720.916772:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04b22 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.916781:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04b22 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.916933:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04b22 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.916943:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04b22 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.917091:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493678080:12345-192.168.203.60@tcp:4:dd.0 Request processed in 73703us (74268us total) trans 4294972767 rc 0/0 00000100:00100000:1.0:1789760720.917101:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5478 00000100:00100000:1.0:1789760720.917121:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731106978412 time 731184405712 state 0x2 ref 0 00000100:00100000:2.0:1789760720.922029:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760720.922119:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493678208 00000100:00100000:3.0:1789760720.922130:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493678208 00000100:00100000:3.0:1789760720.922189:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731106978412 time 731184405712 state 0x0 ref 1 00000100:00100000:3.0:1789760720.922206:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731184666128 time 731184405712 state 0x1 ref 1 00000100:00100000:3.0:1789760720.922275:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731184666128 time 731263310641 state 0x0 ref 1 00000100:00100000:3.0:1789760720.922288:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493678208:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760720.922337:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22319104, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:3.0:1789760720.922345:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b29 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.922362:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760720.922366:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760720.922372:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.922387:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760720.970588:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972768 is committed 00002000:00010000:3.0:1789760720.970679:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22319104 -> 22323200 00010000:00010000:3.0:1789760720.970754:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.970767:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b29 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.970777:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b29 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.970785:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760720.970789:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b29 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.970928:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b29 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.970950:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b29 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760720.970957:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b29 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760720.971106:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493678208:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48815us (49082us total) trans 4294972768 rc 0/0 00000100:00100000:3.0:1789760720.971191:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5479 00000100:00100000:3.0:1789760720.971201:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731184666128 time 731263310641 state 0x2 ref 0 00000100:00100000:2.0:1789760720.974520:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760720.974611:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493678336 00000100:00100000:1.0:1789760720.974620:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493678336 00000100:00100000:1.0:1789760720.974736:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731184666128 time 731263310641 state 0x0 ref 1 00000100:00100000:1.0:1789760720.974753:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731263571057 time 731263310641 state 0x1 ref 1 00000100:00100000:1.0:1789760720.974917:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731263571057 time 731315951216 state 0x0 ref 1 00000100:00100000:1.0:1789760720.974930:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493678336:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760720.975032:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22323200, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:1.0:1789760720.975042:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b30 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.975058:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760720.975062:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.975068:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.975081:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760721.014552:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972769 is committed 00002000:00010000:1.0:1789760721.015026:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22323200 -> 22327296 00010000:00010000:1.0:1789760721.015042:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.015054:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b30 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.015063:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b30 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.015070:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760721.015075:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b30 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.015084:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b30 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.015103:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b30 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.015121:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b30 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.015214:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493678336:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40280us (40697us total) trans 4294972769 rc 0/0 00000100:00100000:1.0:1789760721.015228:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5480 00000100:00100000:1.0:1789760721.015238:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731263571057 time 731315951216 state 0x2 ref 0 00000100:00100000:2.0:1789760721.026100:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760721.026433:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493678464 00000100:00100000:3.0:1789760721.026443:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493678464 00000100:00100000:3.0:1789760721.026511:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731263571057 time 731315951216 state 0x0 ref 1 00000100:00100000:3.0:1789760721.026524:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731316211632 time 731315951216 state 0x1 ref 1 00000100:00100000:3.0:1789760721.026549:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731316211632 time 731367585068 state 0x0 ref 1 00000100:00100000:3.0:1789760721.026561:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493678464:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760721.026622:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22327296, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:3.0:1789760721.026639:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b37 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.026655:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760721.026659:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760721.026665:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.026678:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760721.026935:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760721.078542:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972770 is committed 00002000:00010000:1.0:1789760721.078810:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22327296 -> 22331392 00010000:00010000:1.0:1789760721.078826:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.078839:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b37 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.079321:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b37 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.079331:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760721.079336:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b37 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.079345:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b37 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.079364:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b37 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.079373:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04b37 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.079524:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493678464:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52960us (53430us total) trans 4294972770 rc 0/0 00000100:00100000:1.0:1789760721.079536:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5481 00000100:00100000:1.0:1789760721.079546:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731316211632 time 731367585068 state 0x2 ref 0 00000100:00100000:2.0:1789760721.096405:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760721.096489:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493678592 00000100:00100000:1.0:1789760721.096499:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493678592 00000100:00100000:1.0:1789760721.096553:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731316211632 time 731367585068 state 0x0 ref 1 00000100:00100000:1.0:1789760721.096568:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731367845484 time 731367585068 state 0x1 ref 1 00000100:00100000:1.0:1789760721.096603:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731367845484 time 731437637829 state 0x0 ref 1 00000100:00100000:1.0:1789760721.096616:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493678592:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760721.096675:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22331392, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:1.0:1789760721.096685:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b3e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.096701:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760721.096705:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.096711:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.096725:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760721.148353:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972771 is committed 00002000:00010000:1.0:1789760721.148464:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22331392 -> 22335488 00010000:00010000:1.0:1789760721.148478:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.148486:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b3e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.148492:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b3e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.148496:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760721.148500:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b3e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.148506:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b3e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.148521:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b3e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.148526:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b3e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.148645:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493678592:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52027us (52244us total) trans 4294972771 rc 0/0 00000100:00100000:1.0:1789760721.148660:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5482 00000100:00100000:1.0:1789760721.148670:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731367845484 time 731437637829 state 0x2 ref 0 00000100:00100000:2.0:1789760721.155305:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760721.156059:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493678720 00000100:00100000:3.0:1789760721.156078:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493678720 00000100:00100000:3.0:1789760721.156179:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731367845484 time 731437637829 state 0x0 ref 1 00000100:00100000:3.0:1789760721.156198:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731437898245 time 731437637829 state 0x1 ref 1 00000100:00100000:3.0:1789760721.156235:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731437898245 time 731497270704 state 0x0 ref 1 00000100:00100000:3.0:1789760721.156250:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493678720:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760721.156327:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22335488, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:3.0:1789760721.156340:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04b45 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.156361:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760721.156366:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760721.156374:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04b45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.156392:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04b45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760721.230146:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972772 is committed 00002000:00010000:1.0:1789760721.230493:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22335488 -> 22339584 00010000:00010000:1.0:1789760721.230510:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04b45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.230524:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04b45 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.230534:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04b45 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.230541:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760721.230547:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04b45 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.230557:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04b45 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.230578:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04b45 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.230586:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04b45 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.230714:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493678720:12345-192.168.203.60@tcp:4:dd.0 Request processed in 74463us (75414us total) trans 4294972772 rc 0/0 00000100:00100000:1.0:1789760721.230726:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5483 00000100:00100000:1.0:1789760721.230883:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731437898245 time 731497270704 state 0x2 ref 0 00000100:00100000:0.0:1789760721.235116:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760721.235274:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493678848 00000100:00100000:1.0:1789760721.235286:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493678848 00000100:00100000:1.0:1789760721.235347:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731437898245 time 731497270704 state 0x0 ref 1 00000100:00100000:1.0:1789760721.235362:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731497531120 time 731497270704 state 0x1 ref 1 00000100:00100000:1.0:1789760721.235396:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731497531120 time 731576431080 state 0x0 ref 1 00000100:00100000:1.0:1789760721.235409:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc17480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493678848:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760721.235461:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22339584, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:1.0:1789760721.235470:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b4c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.235487:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760721.235491:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.235498:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.235513:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760721.307929:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972773 is committed 00002000:00010000:1.0:1789760721.308029:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22339584 -> 22343680 00010000:00010000:1.0:1789760721.308045:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.308059:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b4c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.308068:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b4c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.308076:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760721.308080:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b4c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.308089:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b4c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.308110:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b4c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.308118:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b4c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.308254:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc17480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493678848:12345-192.168.203.60@tcp:4:dd.0 Request processed in 72843us (73145us total) trans 4294972773 rc 0/0 00000100:00100000:1.0:1789760721.308269:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5484 00000100:00100000:1.0:1789760721.308280:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731497531120 time 731576431080 state 0x2 ref 0 00000100:00100000:3.0:1789760721.318139:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760721.318560:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493678976 00000100:00100000:1.0:1789760721.318570:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493678976 00000100:00100000:1.0:1789760721.319057:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731497531120 time 731576431080 state 0x0 ref 1 00000100:00100000:1.0:1789760721.319074:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731576691496 time 731576431080 state 0x1 ref 1 00000100:00100000:1.0:1789760721.319164:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731576691496 time 731660197867 state 0x0 ref 1 00000100:00100000:1.0:1789760721.319180:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493678976:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760721.319232:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22343680, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:1.0:1789760721.319242:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b53 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.319258:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760721.319263:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.319270:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.319284:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760721.356830:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972774 is committed 00002000:00010000:1.0:1789760721.356919:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22343680 -> 22347776 00010000:00010000:1.0:1789760721.356933:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.356946:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b53 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.356955:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b53 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.357029:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760721.357034:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b53 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.357043:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b53 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.357062:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b53 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.357069:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b53 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.357187:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493678976:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38008us (39054us total) trans 4294972774 rc 0/0 00000100:00100000:1.0:1789760721.357197:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5485 00000100:00100000:1.0:1789760721.357207:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731576691496 time 731660197867 state 0x2 ref 0 00000100:00100000:2.0:1789760721.371388:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760721.374274:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493679104 00000100:00100000:1.0:1789760721.374291:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493679104 00000100:00100000:1.0:1789760721.374364:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731576691496 time 731660197867 state 0x0 ref 1 00000100:00100000:1.0:1789760721.374380:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731660458283 time 731660197867 state 0x1 ref 1 00000100:00100000:1.0:1789760721.374407:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731660458283 time 731715442797 state 0x0 ref 1 00000100:00100000:1.0:1789760721.374421:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493679104:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760721.374474:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22347776, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:1.0:1789760721.374484:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b5a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.374500:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760721.374505:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.374512:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.374526:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.467084:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760721.497895:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22347776 -> 22351872 00010000:00010000:1.0:1789760721.497916:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.497929:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b5a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.497944:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.497952:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760721.497957:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.498002:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b5a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.498021:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b5a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.498030:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04b5a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.498136:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493679104:12345-192.168.203.60@tcp:4:dd.0 Request processed in 123712us (126753us total) trans 4294972775 rc 0/0 00000100:00100000:1.0:1789760721.498148:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5486 00000100:00100000:1.0:1789760721.498158:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731660458283 time 731715442797 state 0x2 ref 0 00000001:00080000:0.0:1789760721.498589:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972775 is committed 00000100:00100000:1.0:1789760721.505475:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760721.505556:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493679232 00000100:00100000:3.0:1789760721.505567:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493679232 00000100:00100000:3.0:1789760721.505636:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731660458283 time 731715442797 state 0x0 ref 1 00000100:00100000:3.0:1789760721.505651:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731715703213 time 731715442797 state 0x1 ref 1 00000100:00100000:3.0:1789760721.505687:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731715703213 time 731846714354 state 0x0 ref 1 00000100:00100000:3.0:1789760721.505700:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493679232:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760721.505825:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22351872, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:3.0:1789760721.505836:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04b61 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.505851:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760721.505857:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760721.505863:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04b61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.505877:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04b61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760721.561286:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22351872 -> 22355968 00010000:00010000:3.0:1789760721.561305:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04b61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.561317:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04b61 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.561327:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04b61 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.561335:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760721.561340:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04b61 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.561351:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04b61 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.561371:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04b61 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.561380:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04b61 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760721.561471:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493679232:12345-192.168.203.60@tcp:4:dd.0 Request processed in 55768us (56001us total) trans 4294972776 rc 0/0 00000100:00100000:3.0:1789760721.561481:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5487 00000100:00100000:3.0:1789760721.561491:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731715703213 time 731846714354 state 0x2 ref 0 00000001:00080000:0.0:1789760721.561569:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972776 is committed 00000100:00100000:1.0:1789760721.576067:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760721.576364:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493679360 00000100:00100000:0.0:1789760721.576374:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493679360 00000100:00100000:0.0:1789760721.576431:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731715703213 time 731846714354 state 0x0 ref 1 00000100:00100000:0.0:1789760721.576445:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731846974770 time 731846714354 state 0x1 ref 1 00000100:00100000:0.0:1789760721.576515:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731846974770 time 731917549694 state 0x0 ref 1 00000100:00100000:0.0:1789760721.576527:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493679360:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760721.576577:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22355968, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:0.0:1789760721.576587:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04b68 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.576602:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760721.576606:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760721.576613:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04b68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.576638:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04b68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760721.608228:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972777 is committed 00002000:00010000:0.0:1789760721.608450:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22355968 -> 22360064 00010000:00010000:0.0:1789760721.608466:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04b68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.608478:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04b68 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.608486:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04b68 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.608492:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760721.608497:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04b68 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.608504:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04b68 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.608614:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04b68 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.608624:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04b68 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760721.608760:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493679360:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32230us (32698us total) trans 4294972777 rc 0/0 00000100:00100000:0.0:1789760721.608775:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5488 00000100:00100000:0.0:1789760721.608784:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731846974770 time 731917549694 state 0x2 ref 0 00000100:00100000:1.0:1789760721.613612:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760721.613705:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493679488 00000100:00100000:3.0:1789760721.613715:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493679488 00000100:00100000:3.0:1789760721.613769:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731846974770 time 731917549694 state 0x0 ref 1 00000100:00100000:3.0:1789760721.613785:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731917810110 time 731917549694 state 0x1 ref 1 00000100:00100000:3.0:1789760721.613813:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731917810110 time 731954848630 state 0x0 ref 1 00000100:00100000:3.0:1789760721.613825:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493679488:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760721.613875:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22360064, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0xb007 00010000:00010000:3.0:1789760721.613884:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04b6f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.613909:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760721.613915:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760721.613921:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04b6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760721.613936:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04b6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760721.675281:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972778 is committed 00002000:00010000:2.0:1789760721.675400:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22360064 -> 22364160 00002000:00010000:2.0:1789760721.675405:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760720 -> 1789760721 00002000:00010000:2.0:1789760721.675408:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760720 -> 1789760721 00010000:00010000:2.0:1789760721.675422:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04b6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.675434:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04b6f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.675443:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04b6f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.675451:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760721.675456:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04b6f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.675465:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04b6f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.675489:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04b6f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.675498:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04b6f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760721.675605:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493679488:12345-192.168.203.60@tcp:4:dd.0 Request processed in 61769us (61989us total) trans 4294972778 rc 0/0 00000100:00100000:2.0:1789760721.675618:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5489 00000100:00100000:2.0:1789760721.675637:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731917810110 time 731954848630 state 0x2 ref 0 00000100:00100000:1.0:1789760721.696687:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760721.696952:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493679616 00000100:00100000:2.0:1789760721.697005:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493679616 00000100:00100000:2.0:1789760721.697067:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731917810110 time 731954848630 state 0x0 ref 1 00000100:00100000:2.0:1789760721.697084:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731955109046 time 731954848630 state 0x1 ref 1 00000100:00100000:2.0:1789760721.697108:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731955109046 time 732038144082 state 0x0 ref 1 00000100:00100000:2.0:1789760721.697122:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493679616:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760721.697172:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22364160, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:2.0:1789760721.697182:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04b76 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.697197:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760721.697202:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760721.697209:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04b76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.697223:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04b76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760721.737499:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22364160 -> 22368256 00010000:00010000:2.0:1789760721.737519:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04b76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.737533:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04b76 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.737542:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04b76 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.737549:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760721.737554:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04b76 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.737563:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04b76 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.737583:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04b76 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.737590:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04b76 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760721.737695:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493679616:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40570us (41020us total) trans 4294972779 rc 0/0 00000100:00100000:2.0:1789760721.737705:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5490 00000100:00100000:2.0:1789760721.737715:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731955109046 time 732038144082 state 0x2 ref 0 00000001:00080000:1.0:1789760721.739917:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972779 is committed 00000100:00100000:1.0:1789760721.741317:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760721.741411:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493679744 00000100:00100000:0.0:1789760721.741420:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493679744 00000100:00100000:0.0:1789760721.741474:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 731955109046 time 732038144082 state 0x0 ref 1 00000100:00100000:0.0:1789760721.741490:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732038404498 time 732038144082 state 0x1 ref 1 00000100:00100000:0.0:1789760721.741553:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732038404498 time 732082588023 state 0x0 ref 1 00000100:00100000:0.0:1789760721.741567:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493679744:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760721.741617:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22368256, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:0.0:1789760721.741635:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04b7d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.741651:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760721.741655:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760721.741661:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04b7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.741674:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04b7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760721.792552:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972780 is committed 00002000:00010000:1.0:1789760721.792718:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22368256 -> 22372352 00010000:00010000:1.0:1789760721.792733:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04b7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.792746:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04b7d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.792755:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04b7d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.792763:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760721.792768:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04b7d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.792776:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04b7d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.792807:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04b7d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.792814:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04b7d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.793165:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493679744:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51595us (51852us total) trans 4294972780 rc 0/0 00000100:00100000:1.0:1789760721.793179:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5491 00000100:00100000:1.0:1789760721.793189:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732038404498 time 732082588023 state 0x2 ref 0 00000100:00100000:0.0:1789760721.797505:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760721.797583:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493679872 00000100:00100000:1.0:1789760721.797603:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493679872 00000100:00100000:1.0:1789760721.797673:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732038404498 time 732082588023 state 0x0 ref 1 00000100:00100000:1.0:1789760721.797699:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732082848439 time 732082588023 state 0x1 ref 1 00000100:00100000:1.0:1789760721.797728:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732082848439 time 732138763701 state 0x0 ref 1 00000100:00100000:1.0:1789760721.797742:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493679872:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760721.797805:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22372352, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:1.0:1789760721.797815:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04b84 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.797832:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760721.797837:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.798258:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04b84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.798277:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04b84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760721.860498:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22372352 -> 22376448 00010000:00010000:2.0:1789760721.860518:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04b84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.860534:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04b84 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.860545:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04b84 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.860558:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760721.860563:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04b84 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.860573:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04b84 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.860673:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04b84 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.860749:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04b84 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760721.861141:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493679872:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63396us (63643us total) trans 4294972781 rc 0/0 00000100:00100000:2.0:1789760721.864548:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5492 00000100:00100000:2.0:1789760721.864563:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732082848439 time 732138763701 state 0x2 ref 0 00000001:00080000:3.0:1789760721.868150:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972781 is committed 00000100:00100000:2.0:1789760721.874911:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760721.875003:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493680000 00000100:00100000:1.0:1789760721.875014:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493680000 00000100:00100000:1.0:1789760721.875081:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732082848439 time 732138763701 state 0x0 ref 1 00000100:00100000:1.0:1789760721.875095:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732139024117 time 732138763701 state 0x1 ref 1 00000100:00100000:1.0:1789760721.875127:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732139024117 time 732216163095 state 0x0 ref 1 00000100:00100000:1.0:1789760721.875140:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8556a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493680000:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760721.875188:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22376448, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:1.0:1789760721.875198:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04b8b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.875212:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760721.875216:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.875223:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04b8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.875237:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04b8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760721.916696:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972782 is committed 00002000:00010000:0.0:1789760721.916885:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22376448 -> 22380544 00010000:00010000:0.0:1789760721.916902:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04b8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.916916:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04b8b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.916924:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04b8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.916931:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760721.916936:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04b8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.916945:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04b8b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.916992:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04b8b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.917005:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04b8b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760721.917093:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8556a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493680000:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41950us (42186us total) trans 4294972782 rc 0/0 00000100:00100000:0.0:1789760721.917117:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5493 00000100:00100000:0.0:1789760721.917128:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732139024117 time 732216163095 state 0x2 ref 0 00000100:00100000:1.0:1789760721.920696:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760721.920832:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493680128 00000100:00100000:2.0:1789760721.920845:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493680128 00000100:00100000:2.0:1789760721.920907:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732139024117 time 732216163095 state 0x0 ref 1 00000100:00100000:2.0:1789760721.920923:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732216423511 time 732216163095 state 0x1 ref 1 00000100:00100000:2.0:1789760721.920955:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732216423511 time 732261990347 state 0x0 ref 1 00000100:00100000:2.0:1789760721.921003:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493680128:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760721.921053:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22380544, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:2.0:1789760721.921061:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04b92 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.921077:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760721.921081:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760721.921088:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04b92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.921193:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04b92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760721.955172:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22380544 -> 22384640 00010000:00010000:1.0:1789760721.955192:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04b92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.955206:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04b92 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.955221:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04b92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.955228:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760721.955234:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04b92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.955242:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04b92 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.955260:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04b92 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.955268:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04b92 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.955361:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493680128:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34354us (34680us total) trans 4294972783 rc 0/0 00000100:00100000:1.0:1789760721.955371:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5494 00000100:00100000:1.0:1789760721.955381:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732216423511 time 732261990347 state 0x2 ref 0 00000001:00080000:3.0:1789760721.955508:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972783 is committed 00000100:00100000:1.0:1789760721.959181:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760721.959265:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493680256 00000100:00100000:0.0:1789760721.959277:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493680256 00000100:00100000:0.0:1789760721.959335:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732216423511 time 732261990347 state 0x0 ref 1 00000100:00100000:0.0:1789760721.959350:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732262250763 time 732261990347 state 0x1 ref 1 00000100:00100000:0.0:1789760721.959383:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732262250763 time 732300418896 state 0x0 ref 1 00000100:00100000:0.0:1789760721.959396:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493680256:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760721.959448:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22384640, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:0.0:1789760721.959458:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04b99 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.959474:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760721.959478:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760721.959484:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04b99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.959499:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04b99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760721.987447:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972784 is committed 00002000:00010000:0.0:1789760721.987562:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22384640 -> 22388736 00010000:00010000:0.0:1789760721.987579:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04b99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.987593:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04b99 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.987603:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04b99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.987611:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760721.987616:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04b99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.987635:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04b99 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.987657:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04b99 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.987665:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04b99 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760721.987894:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493680256:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28492us (28715us total) trans 4294972784 rc 0/0 00000100:00100000:0.0:1789760721.987908:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5495 00000100:00100000:0.0:1789760721.987920:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732262250763 time 732300418896 state 0x2 ref 0 00000100:00100000:0.0:1789760721.991485:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760721.991593:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493680384 00000100:00100000:1.0:1789760721.991604:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493680384 00000100:00100000:1.0:1789760721.991742:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732262250763 time 732300418896 state 0x0 ref 1 00000100:00100000:1.0:1789760721.991761:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732300679312 time 732300418896 state 0x1 ref 1 00000100:00100000:1.0:1789760721.991831:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732300679312 time 732332865159 state 0x0 ref 1 00000100:00100000:1.0:1789760721.991845:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493680384:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760721.991896:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22388736, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:1.0:1789760721.991906:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04ba0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.991923:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760721.991927:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.991935:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04ba0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.991949:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04ba0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760722.017312:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972785 is committed 00002000:00010000:1.0:1789760722.017414:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22388736 -> 22392832 00010000:00010000:1.0:1789760722.017431:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04ba0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.017444:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04ba0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.017453:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04ba0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.017461:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760722.017467:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04ba0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.017477:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04ba0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.017496:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04ba0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.017503:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04ba0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.017592:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493680384:12345-192.168.203.60@tcp:4:dd.0 Request processed in 25745us (26115us total) trans 4294972785 rc 0/0 00000100:00100000:1.0:1789760722.017603:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5496 00000100:00100000:1.0:1789760722.017613:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732300679312 time 732332865159 state 0x2 ref 0 00000100:00100000:2.0:1789760722.023214:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760722.023289:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760722.023317:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760722.023338:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760722.023364:0:10519:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493680896 00000100:00100000:2.0:1789760722.023373:0:10519:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493680896 00000100:00100000:2.0:1789760722.023422:0:10519:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493680768 00000100:00100000:2.0:1789760722.023424:0:10519:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493680768 00000100:00100000:2.0:1789760722.023445:0:10519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 462 00000100:00100000:2.0:1789760722.023453:0:10519:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8555500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2370:x1876699493680896:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:2.0:1789760722.023521:0:10519:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8555500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2370:x1876699493680896:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 69us (203us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760722.023528:0:10519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 462 00000100:00100000:2.0:1789760722.023545:0:10519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 463 00000100:00100000:2.0:1789760722.023552:0:10519:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8557b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2370:x1876699493680768:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:2.0:1789760722.023574:0:10519:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8557b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2370:x1876699493680768:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 23us (234us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760722.023580:0:10519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 463 00000100:00080000:2.0:1789760722.023594:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760722.023761:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493680640 00000100:00100000:3.0:1789760722.023765:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493680640 00000100:00100000:3.0:1789760722.023794:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 391 00000100:00100000:3.0:1789760722.023800:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d4f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2370:x1876699493680640:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:3.0:1789760722.023826:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d4f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2370:x1876699493680640:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 26us (538us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760722.023832:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 391 00000100:00100000:1.0:1789760722.024921:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493680512 00000100:00100000:1.0:1789760722.024930:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493680512 00000100:00100000:1.0:1789760722.025007:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732300679312 time 732332865159 state 0x0 ref 1 00000100:00100000:1.0:1789760722.025021:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732333125575 time 732332865159 state 0x1 ref 1 00000100:00100000:1.0:1789760722.025041:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732333125575 time 732366076708 state 0x0 ref 1 00000100:00100000:1.0:1789760722.025056:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493680512:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760722.025099:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22392832, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:1.0:1789760722.025210:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04ba7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.025226:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760722.025231:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.025237:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04ba7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.025252:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04ba7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760722.059919:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22392832 -> 22396928 00010000:00010000:1.0:1789760722.059940:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04ba7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.059953:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04ba7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.059987:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04ba7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.059994:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760722.059999:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04ba7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.060009:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04ba7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.060029:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04ba7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.060037:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e04ba7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.060344:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493680512:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35284us (37134us total) trans 4294972786 rc 0/0 00000100:00100000:1.0:1789760722.060355:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5497 00000100:00100000:1.0:1789760722.060365:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732333125575 time 732366076708 state 0x2 ref 0 00000001:00080000:0.0:1789760722.061607:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972786 is committed 00000100:00100000:1.0:1789760722.064174:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760722.064315:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493681024 00000100:00100000:3.0:1789760722.064325:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493681024 00000100:00100000:3.0:1789760722.064382:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732333125575 time 732366076708 state 0x0 ref 1 00000100:00100000:3.0:1789760722.064397:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732366337124 time 732366076708 state 0x1 ref 1 00000100:00100000:3.0:1789760722.064427:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732366337124 time 732405462347 state 0x0 ref 1 00000100:00100000:3.0:1789760722.064443:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb07450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493681024:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760722.064488:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22396928, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:3.0:1789760722.064497:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04bae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.064513:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760722.064518:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.064526:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04bae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.064539:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04bae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.111637:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972787 is committed 00002000:00010000:3.0:1789760722.112444:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22396928 -> 22401024 00010000:00010000:3.0:1789760722.112459:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04bae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.112471:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04bae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.112479:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04bae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.112485:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760722.112489:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04bae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.112497:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04bae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.112517:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04bae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.112523:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04bae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.112616:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb07450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493681024:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48170us (48446us total) trans 4294972787 rc 0/0 00000100:00100000:3.0:1789760722.112635:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5498 00000100:00100000:3.0:1789760722.112645:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732366337124 time 732405462347 state 0x2 ref 0 00000100:00100000:3.0:1789760722.116914:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760722.118377:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493681152 00000100:00100000:0.0:1789760722.118392:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493681152 00000100:00100000:0.0:1789760722.118458:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732366337124 time 732405462347 state 0x0 ref 1 00000100:00100000:0.0:1789760722.118474:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732405722763 time 732405462347 state 0x1 ref 1 00000100:00100000:0.0:1789760722.118510:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732405722763 time 732459545210 state 0x0 ref 1 00000100:00100000:0.0:1789760722.118523:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493681152:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760722.118573:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22401024, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:0.0:1789760722.118581:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04bb5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.118598:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760722.118603:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.118610:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04bb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.118633:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04bb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.184456:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972788 is committed 00002000:00010000:3.0:1789760722.184650:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22401024 -> 22405120 00010000:00010000:3.0:1789760722.184665:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04bb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.184687:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04bb5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.184696:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04bb5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.184707:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760722.184712:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04bb5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.184720:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04bb5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.184738:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04bb5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.184746:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04bb5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.184838:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493681152:12345-192.168.203.60@tcp:4:dd.0 Request processed in 66313us (67998us total) trans 4294972788 rc 0/0 00000100:00100000:3.0:1789760722.184848:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5499 00000100:00100000:3.0:1789760722.184858:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732405722763 time 732459545210 state 0x2 ref 0 00000100:00100000:2.0:1789760722.188715:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760722.188815:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493681280 00000100:00100000:1.0:1789760722.188824:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493681280 00000100:00100000:1.0:1789760722.188892:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732405722763 time 732459545210 state 0x0 ref 1 00000100:00100000:1.0:1789760722.188909:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732459805626 time 732459545210 state 0x1 ref 1 00000100:00100000:1.0:1789760722.188939:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732459805626 time 732529974717 state 0x0 ref 1 00000100:00100000:1.0:1789760722.188951:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8555180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493681280:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760722.189033:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22405120, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:1.0:1789760722.189043:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04bbc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.189058:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760722.189062:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.189069:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04bbc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.189089:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04bbc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760722.250088:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972789 is committed 00002000:00010000:3.0:1789760722.250224:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22405120 -> 22409216 00010000:00010000:3.0:1789760722.250239:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04bbc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.250251:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04bbc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.250265:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04bbc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.250272:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760722.250276:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04bbc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.250285:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04bbc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.250304:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04bbc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.250311:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04bbc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.250421:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8555180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493681280:12345-192.168.203.60@tcp:4:dd.0 Request processed in 61456us (61699us total) trans 4294972789 rc 0/0 00000100:00100000:3.0:1789760722.250431:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5500 00000100:00100000:3.0:1789760722.250440:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732459805626 time 732529974717 state 0x2 ref 0 00000100:00100000:1.0:1789760722.255806:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760722.255904:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493681408 00000100:00100000:0.0:1789760722.255915:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493681408 00000100:00100000:0.0:1789760722.256013:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732459805626 time 732529974717 state 0x0 ref 1 00000100:00100000:0.0:1789760722.256028:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732530235133 time 732529974717 state 0x1 ref 1 00000100:00100000:0.0:1789760722.256055:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732530235133 time 732597090595 state 0x0 ref 1 00000100:00100000:0.0:1789760722.256075:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493681408:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760722.256136:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22409216, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:0.0:1789760722.256145:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04bc3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.256161:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760722.256165:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.256171:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04bc3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.256185:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04bc3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.302145:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972790 is committed 00002000:00010000:0.0:1789760722.302385:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22409216 -> 22413312 00010000:00010000:0.0:1789760722.302400:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04bc3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.302412:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04bc3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.302420:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04bc3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.302427:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760722.302431:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04bc3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.302441:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04bc3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.302459:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04bc3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.302468:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04bc3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760722.302558:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493681408:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46483us (46757us total) trans 4294972790 rc 0/0 00000100:00100000:0.0:1789760722.302567:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5501 00000100:00100000:0.0:1789760722.302577:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732530235133 time 732597090595 state 0x2 ref 0 00000100:00100000:3.0:1789760722.306901:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760722.307061:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493681536 00000100:00100000:0.0:1789760722.307072:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493681536 00000100:00100000:0.0:1789760722.307136:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732530235133 time 732597090595 state 0x0 ref 1 00000100:00100000:0.0:1789760722.307151:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732597351011 time 732597090595 state 0x1 ref 1 00000100:00100000:0.0:1789760722.307220:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732597351011 time 732648255216 state 0x0 ref 1 00000100:00100000:0.0:1789760722.307234:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493681536:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760722.307284:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22413312, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:0.0:1789760722.307294:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04bca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.307311:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760722.307316:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.307322:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04bca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.307336:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04bca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760722.352161:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22413312 -> 22417408 00010000:00010000:0.0:1789760722.352183:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04bca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.352195:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04bca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.352204:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04bca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.352212:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760722.352216:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04bca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.352224:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04bca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.352255:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04bca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.352262:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04bca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760722.352358:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493681536:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45122us (45462us total) trans 4294972791 rc 0/0 00000100:00100000:0.0:1789760722.352368:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5502 00000100:00100000:0.0:1789760722.352379:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732597351011 time 732648255216 state 0x2 ref 0 00000001:00080000:3.0:1789760722.359336:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972791 is committed 00000100:00100000:0.0:1789760722.359454:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760722.359692:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493681664 00000100:00100000:3.0:1789760722.359702:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493681664 00000100:00100000:3.0:1789760722.359752:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732597351011 time 732648255216 state 0x0 ref 1 00000100:00100000:3.0:1789760722.359766:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732648515632 time 732648255216 state 0x1 ref 1 00000100:00100000:3.0:1789760722.359792:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732648515632 time 732700827955 state 0x0 ref 1 00000100:00100000:3.0:1789760722.359803:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493681664:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760722.359846:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22417408, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:3.0:1789760722.359855:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04bd1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.359870:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760722.359874:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.359880:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04bd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.359892:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04bd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.389521:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972792 is committed 00002000:00010000:0.0:1789760722.391192:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22417408 -> 22421504 00010000:00010000:0.0:1789760722.391211:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04bd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.391226:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04bd1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.391239:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04bd1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.391247:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760722.391252:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04bd1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.391261:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04bd1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.391290:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04bd1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.391297:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04bd1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760722.391401:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493681664:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31593us (31945us total) trans 4294972792 rc 0/0 00000100:00100000:0.0:1789760722.391412:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5503 00000100:00100000:0.0:1789760722.391422:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732648515632 time 732700827955 state 0x2 ref 0 00000100:00100000:1.0:1789760722.394776:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760722.394859:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493681792 00000100:00100000:3.0:1789760722.394868:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493681792 00000100:00100000:3.0:1789760722.394923:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732648515632 time 732700827955 state 0x0 ref 1 00000100:00100000:3.0:1789760722.394940:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732701088371 time 732700827955 state 0x1 ref 1 00000100:00100000:0.0:1789760722.395077:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732701088371 time 732736109724 state 0x0 ref 1 00000100:00100000:0.0:1789760722.395091:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493681792:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760722.395150:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22421504, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:0.0:1789760722.395159:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04bd8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.395176:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760722.395180:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.395187:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04bd8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.395201:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04bd8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.423778:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972793 is committed 00002000:00010000:0.0:1789760722.423829:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22421504 -> 22425600 00010000:00010000:0.0:1789760722.423847:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04bd8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.423860:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04bd8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.423867:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04bd8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.423871:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760722.423875:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04bd8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.423880:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04bd8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.423896:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04bd8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.423901:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04bd8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760722.424049:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493681792:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28950us (29273us total) trans 4294972793 rc 0/0 00000100:00100000:0.0:1789760722.424063:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5504 00000100:00100000:0.0:1789760722.424072:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732701088371 time 732736109724 state 0x2 ref 0 00000100:00100000:2.0:1789760722.426484:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760722.426551:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493681920 00000100:00100000:0.0:1789760722.426560:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493681920 00000100:00100000:0.0:1789760722.426610:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732701088371 time 732736109724 state 0x0 ref 1 00000100:00100000:0.0:1789760722.426638:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732736370140 time 732736109724 state 0x1 ref 1 00000100:00100000:0.0:1789760722.426671:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732736370140 time 732767706828 state 0x0 ref 1 00000100:00100000:0.0:1789760722.426690:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493681920:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760722.426746:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22425600, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:0.0:1789760722.426755:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04bdf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.426772:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760722.426776:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.426783:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04bdf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.426796:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04bdf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.456909:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972794 is committed 00002000:00010000:1.0:1789760722.457002:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22425600 -> 22429696 00010000:00010000:1.0:1789760722.457017:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04bdf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.457031:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04bdf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.457040:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04bdf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.457053:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760722.457058:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04bdf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.457068:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04bdf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.457090:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04bdf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.457098:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04bdf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.457335:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493681920:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30641us (30854us total) trans 4294972794 rc 0/0 00000100:00100000:1.0:1789760722.457346:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5505 00000100:00100000:1.0:1789760722.457356:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732736370140 time 732767706828 state 0x2 ref 0 00000100:00100000:2.0:1789760722.474788:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760722.474857:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493682048 00000100:00100000:1.0:1789760722.474868:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493682048 00000100:00100000:1.0:1789760722.474925:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732736370140 time 732767706828 state 0x0 ref 1 00000100:00100000:1.0:1789760722.474941:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732767967244 time 732767706828 state 0x1 ref 1 00000100:00100000:1.0:1789760722.474996:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732767967244 time 732816030960 state 0x0 ref 1 00000100:00100000:1.0:1789760722.475012:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8556a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493682048:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760722.475060:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22429696, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:1.0:1789760722.475069:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04be6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.475085:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760722.475090:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.475095:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04be6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.475108:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04be6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.490225:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760722.516520:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972795 is committed 00002000:00010000:1.0:1789760722.516600:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22429696 -> 22433792 00010000:00010000:1.0:1789760722.516616:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04be6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.516652:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04be6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.516662:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04be6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.516669:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760722.516682:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04be6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.516693:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04be6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.517408:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04be6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.517419:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04be6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.517517:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8556a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493682048:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42501us (42732us total) trans 4294972795 rc 0/0 00000100:00100000:1.0:1789760722.517529:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5506 00000100:00100000:1.0:1789760722.517540:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732767967244 time 732816030960 state 0x2 ref 0 00000100:00100000:2.0:1789760722.524630:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760722.525052:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493682176 00000100:00100000:3.0:1789760722.525062:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493682176 00000100:00100000:3.0:1789760722.525113:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732767967244 time 732816030960 state 0x0 ref 1 00000100:00100000:3.0:1789760722.525129:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732816291376 time 732816030960 state 0x1 ref 1 00000100:00100000:3.0:1789760722.525202:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732816291376 time 732866236947 state 0x0 ref 1 00000100:00100000:3.0:1789760722.525215:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493682176:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760722.525260:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22433792, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:3.0:1789760722.525269:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04bed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.525284:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760722.525288:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.525294:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04bed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.525308:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04bed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760722.568528:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22433792 -> 22437888 00010000:00010000:1.0:1789760722.568549:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04bed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.568563:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04bed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.568572:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04bed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.568580:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760722.568586:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04bed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.568594:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04bed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.568613:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04bed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.568632:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04bed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.568871:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493682176:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43652us (44255us total) trans 4294972796 rc 0/0 00000100:00100000:1.0:1789760722.568882:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5507 00000100:00100000:1.0:1789760722.568891:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732816291376 time 732866236947 state 0x2 ref 0 00000001:00080000:2.0:1789760722.568989:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972796 is committed 00000100:00100000:2.0:1789760722.572842:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760722.572932:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493682304 00000100:00100000:0.0:1789760722.572941:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493682304 00000100:00100000:1.0:1789760722.573161:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732816291376 time 732866236947 state 0x0 ref 1 00000100:00100000:1.0:1789760722.573177:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732866497363 time 732866236947 state 0x1 ref 1 00000100:00100000:1.0:1789760722.573209:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732866497363 time 732914244569 state 0x0 ref 1 00000100:00100000:1.0:1789760722.573223:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493682304:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760722.573274:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22437888, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0xb007 00010000:00010000:1.0:1789760722.573283:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04bf4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.573298:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760722.573303:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.573308:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04bf4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.573321:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04bf4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760722.600723:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972797 is committed 00002000:00010000:1.0:1789760722.601063:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22437888 -> 22441984 00002000:00010000:1.0:1789760722.601069:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760721 -> 1789760722 00002000:00010000:1.0:1789760722.601072:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760721 -> 1789760722 00010000:00010000:1.0:1789760722.601084:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04bf4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.601097:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04bf4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.601107:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04bf4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.601122:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760722.601127:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04bf4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.601137:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04bf4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.601156:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04bf4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.601164:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04bf4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.601437:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493682304:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28212us (28600us total) trans 4294972797 rc 0/0 00000100:00100000:1.0:1789760722.601450:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5508 00000100:00100000:1.0:1789760722.601460:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732866497363 time 732914244569 state 0x2 ref 0 00000100:00100000:0.0:1789760722.606039:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760722.606186:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493682432 00000100:00100000:1.0:1789760722.606197:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493682432 00000100:00100000:1.0:1789760722.606313:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732866497363 time 732914244569 state 0x0 ref 1 00000100:00100000:1.0:1789760722.606332:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732914504985 time 732914244569 state 0x1 ref 1 00000100:00100000:1.0:1789760722.606363:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732914504985 time 732947399840 state 0x0 ref 1 00000100:00100000:1.0:1789760722.606375:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493682432:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760722.606423:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22441984, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb007 00010000:00010000:1.0:1789760722.606432:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04bfb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.606447:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760722.606453:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.606459:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04bfb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.606473:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04bfb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760722.655422:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972798 is committed 00002000:00010000:1.0:1789760722.655524:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22441984 -> 22446080 00010000:00010000:1.0:1789760722.655539:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04bfb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.655551:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04bfb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.655560:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04bfb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.655568:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760722.655573:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04bfb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.655582:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04bfb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.655602:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04bfb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.655610:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04bfb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.655726:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493682432:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49348us (49691us total) trans 4294972798 rc 0/0 00000100:00100000:1.0:1789760722.655737:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5509 00000100:00100000:1.0:1789760722.655747:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732914504985 time 732947399840 state 0x2 ref 0 00000100:00100000:2.0:1789760722.670513:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760722.671888:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493682560 00000100:00100000:1.0:1789760722.671902:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493682560 00000100:00100000:1.0:1789760722.672046:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732914504985 time 732947399840 state 0x0 ref 1 00000100:00100000:1.0:1789760722.672062:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732947660256 time 732947399840 state 0x1 ref 1 00000100:00100000:1.0:1789760722.672218:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732947660256 time 733013252515 state 0x0 ref 1 00000100:00100000:1.0:1789760722.672233:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493682560:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760722.672283:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22446080, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb007 00010000:00010000:1.0:1789760722.672292:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c02 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.672308:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760722.672313:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.672319:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.672334:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760722.732434:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972799 is committed 00002000:00010000:1.0:1789760722.732537:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22446080 -> 22450176 00010000:00010000:1.0:1789760722.732552:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.732565:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c02 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.732574:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.732581:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760722.732585:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.732594:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c02 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.732613:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c02 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.732630:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c02 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.732818:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493682560:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60582us (62309us total) trans 4294972799 rc 0/0 00000100:00100000:1.0:1789760722.732832:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5510 00000100:00100000:1.0:1789760722.732841:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732947660256 time 733013252515 state 0x2 ref 0 00000100:00100000:2.0:1789760722.740984:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760722.741268:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493682688 00000100:00100000:3.0:1789760722.741279:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493682688 00000100:00100000:3.0:1789760722.741333:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 732947660256 time 733013252515 state 0x0 ref 1 00000100:00100000:3.0:1789760722.741349:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733013512931 time 733013252515 state 0x1 ref 1 00000100:00100000:3.0:1789760722.745169:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733013512931 time 733086191439 state 0x0 ref 1 00000100:00100000:3.0:1789760722.745206:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493682688:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760722.745267:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22450176, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb007 00010000:00010000:3.0:1789760722.745277:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04c09 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.745294:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760722.745299:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.745305:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04c09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.745318:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04c09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760722.782693:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972800 is committed 00002000:00010000:3.0:1789760722.782847:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22450176 -> 22454272 00010000:00010000:3.0:1789760722.782861:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04c09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.782874:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04c09 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.782882:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04c09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.782890:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760722.782894:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04c09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.782904:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04c09 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.782925:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04c09 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.782934:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04c09 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.783095:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493682688:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37900us (42128us total) trans 4294972800 rc 0/0 00000100:00100000:3.0:1789760722.783113:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5511 00000100:00100000:3.0:1789760722.783122:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733013512931 time 733086191439 state 0x2 ref 0 00000100:00100000:2.0:1789760722.786487:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760722.786573:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493682816 00000100:00100000:3.0:1789760722.786584:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493682816 00000100:00100000:3.0:1789760722.786649:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733013512931 time 733086191439 state 0x0 ref 1 00000100:00100000:3.0:1789760722.786664:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733086451855 time 733086191439 state 0x1 ref 1 00000100:00100000:3.0:1789760722.786703:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733086451855 time 733127739081 state 0x0 ref 1 00000100:00100000:3.0:1789760722.786716:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493682816:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760722.786769:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22454272, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb007 00010000:00010000:3.0:1789760722.786779:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04c10 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.786795:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760722.786799:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.786806:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04c10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.786819:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04c10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.810458:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789760722.810485:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc5836d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519249152:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760722.810574:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760722.810657:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519249152 00000100:00100000:3.0:1789760722.810666:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519249152 00000100:00100000:3.0:1789760722.810727:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 192 00000100:00100000:3.0:1789760722.810738:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519249152:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789760722.810762:0:6690:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527892480 pending 1703936 free 3835691008 avail 3831496704 00000100:00100000:3.0:1789760722.810808:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519249152:12345-0@lo:13:osp-pre-0-0.0 Request processed in 72us (234us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760722.810815:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 192 00000100:00080000:3.0:1789760722.810833:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760722.810872:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc5836d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519249152:0@lo:13:osp-pre-0-0.0 00000001:00080000:2.0:1789760722.828580:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972801 is committed 00002000:00010000:1.0:1789760722.828830:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22454272 -> 22458368 00010000:00010000:1.0:1789760722.828844:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04c10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.828857:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04c10 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.828865:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04c10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.828872:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760722.828878:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04c10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.828886:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04c10 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.828907:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04c10 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.828913:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04c10 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.829039:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493682816:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42319us (42562us total) trans 4294972801 rc 0/0 00000100:00100000:1.0:1789760722.829049:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5512 00000100:00100000:1.0:1789760722.829059:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733086451855 time 733127739081 state 0x2 ref 0 00000100:00100000:2.0:1789760722.833222:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760722.833307:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493682944 00000100:00100000:1.0:1789760722.833316:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493682944 00000100:00100000:1.0:1789760722.833373:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733086451855 time 733127739081 state 0x0 ref 1 00000100:00100000:1.0:1789760722.833388:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733127999497 time 733127739081 state 0x1 ref 1 00000100:00100000:1.0:1789760722.833416:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733127999497 time 733174451788 state 0x0 ref 1 00000100:00100000:1.0:1789760722.833429:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493682944:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760722.833478:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22458368, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb007 00010000:00010000:1.0:1789760722.833487:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c17 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.833506:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760722.833511:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.833517:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.833531:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760722.869689:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972802 is committed 00002000:00010000:1.0:1789760722.869879:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22458368 -> 22462464 00010000:00010000:1.0:1789760722.869894:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.869906:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c17 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.869916:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c17 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.869924:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760722.869928:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c17 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.869937:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c17 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.869956:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c17 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.869994:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c17 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.870077:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493682944:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36645us (36865us total) trans 4294972802 rc 0/0 00000100:00100000:1.0:1789760722.870091:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5513 00000100:00100000:1.0:1789760722.870100:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733127999497 time 733174451788 state 0x2 ref 0 00000100:00100000:2.0:1789760722.881263:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760722.881420:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493683072 00000100:00100000:1.0:1789760722.881431:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493683072 00000100:00100000:1.0:1789760722.881489:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733127999497 time 733174451788 state 0x0 ref 1 00000100:00100000:1.0:1789760722.881504:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733174712204 time 733174451788 state 0x1 ref 1 00000100:00100000:1.0:1789760722.881534:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733174712204 time 733222570022 state 0x0 ref 1 00000100:00100000:1.0:1789760722.881546:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493683072:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760722.881594:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22462464, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb007 00010000:00010000:1.0:1789760722.881603:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c1e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.881627:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760722.881632:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.881638:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.881651:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760722.919561:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22462464 -> 22466560 00010000:00010000:3.0:1789760722.919593:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.919611:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c1e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.919628:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.919636:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760722.919641:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.919649:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c1e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.919679:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c1e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.919686:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c1e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760722.919785:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493683072:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38235us (38527us total) trans 4294972803 rc 0/0 00000100:00100000:3.0:1789760722.919795:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5514 00000100:00100000:3.0:1789760722.919805:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733174712204 time 733222570022 state 0x2 ref 0 00000001:00080000:0.0:1789760722.919886:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972803 is committed 00000100:00100000:2.0:1789760722.924385:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760722.924465:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493683200 00000100:00100000:3.0:1789760722.924476:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493683200 00000100:00100000:3.0:1789760722.924528:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733174712204 time 733222570022 state 0x0 ref 1 00000100:00100000:3.0:1789760722.924544:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733222830438 time 733222570022 state 0x1 ref 1 00000100:00100000:3.0:1789760722.924576:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733222830438 time 733265610996 state 0x0 ref 1 00000100:00100000:3.0:1789760722.924589:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493683200:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760722.924648:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22466560, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb007 00010000:00010000:3.0:1789760722.924658:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04c25 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.924747:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760722.924753:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.924759:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04c25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.924773:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04c25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760722.958500:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972804 is committed 00002000:00010000:2.0:1789760722.958607:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22466560 -> 22470656 00010000:00010000:2.0:1789760722.958632:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04c25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.958645:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04c25 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.958655:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04c25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.958662:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760722.958666:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04c25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.958681:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04c25 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.958699:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04c25 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.958705:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04c25 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.958797:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493683200:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34206us (34416us total) trans 4294972804 rc 0/0 00000100:00100000:2.0:1789760722.958807:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5515 00000100:00100000:2.0:1789760722.958817:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733222830438 time 733265610996 state 0x2 ref 0 00000100:00100000:2.0:1789760722.962612:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760722.963309:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493683328 00000100:00100000:3.0:1789760722.963320:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493683328 00000100:00100000:3.0:1789760722.963383:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733222830438 time 733265610996 state 0x0 ref 1 00000100:00100000:3.0:1789760722.963399:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733265871412 time 733265610996 state 0x1 ref 1 00000100:00100000:3.0:1789760722.963466:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733265871412 time 733304500625 state 0x0 ref 1 00000100:00100000:3.0:1789760722.963480:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493683328:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760722.963537:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22470656, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb007 00010000:00010000:3.0:1789760722.963545:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04c2c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.963563:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760722.963568:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760722.963575:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04c2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760722.963589:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04c2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.000642:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972805 is committed 00002000:00010000:3.0:1789760723.001029:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22470656 -> 22474752 00010000:00010000:3.0:1789760723.001045:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04c2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.001058:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04c2c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.001066:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04c2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.001073:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760723.001077:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04c2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.001087:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04c2c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.001429:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04c2c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.001438:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04c2c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760723.001578:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493683328:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38095us (38970us total) trans 4294972805 rc 0/0 00000100:00100000:3.0:1789760723.001589:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5516 00000100:00100000:3.0:1789760723.001599:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733265871412 time 733304500625 state 0x2 ref 0 00000100:00100000:3.0:1789760723.008462:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760723.008774:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493683456 00000100:00100000:1.0:1789760723.008787:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493683456 00000100:00100000:1.0:1789760723.008866:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733265871412 time 733304500625 state 0x0 ref 1 00000100:00100000:1.0:1789760723.008881:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733304761041 time 733304500625 state 0x1 ref 1 00000100:00100000:1.0:1789760723.008916:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733304761041 time 733349951183 state 0x0 ref 1 00000100:00100000:1.0:1789760723.008931:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493683456:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760723.009021:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22474752, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb007 00010000:00010000:1.0:1789760723.009030:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04c33 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.009045:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760723.009049:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.009055:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04c33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.009069:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04c33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760723.009988:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760723.052552:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972806 is committed 00002000:00010000:2.0:1789760723.052791:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22474752 -> 22478848 00010000:00010000:2.0:1789760723.052808:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04c33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.052828:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04c33 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.052838:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04c33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.052846:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760723.052851:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04c33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.052861:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04c33 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.052882:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04c33 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.052890:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04c33 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.053021:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493683456:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44089us (44566us total) trans 4294972806 rc 0/0 00000100:00100000:2.0:1789760723.053031:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5517 00000100:00100000:2.0:1789760723.053041:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733304761041 time 733349951183 state 0x2 ref 0 00000100:00100000:2.0:1789760723.057056:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760723.057330:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493683584 00000100:00100000:3.0:1789760723.057340:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493683584 00000100:00100000:3.0:1789760723.057395:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733304761041 time 733349951183 state 0x0 ref 1 00000100:00100000:3.0:1789760723.057411:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733350211599 time 733349951183 state 0x1 ref 1 00000100:00100000:3.0:1789760723.057437:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733350211599 time 733398472824 state 0x0 ref 1 00000100:00100000:3.0:1789760723.057451:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493683584:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760723.057504:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22478848, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb007 00010000:00010000:3.0:1789760723.057514:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04c3a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.057529:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760723.057535:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760723.057541:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04c3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.057555:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04c3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.091071:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972807 is committed 00002000:00010000:2.0:1789760723.091303:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22478848 -> 22482944 00010000:00010000:2.0:1789760723.091319:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04c3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.091333:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04c3a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.091343:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04c3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.091350:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760723.091355:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04c3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.091366:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04c3a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.091386:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04c3a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.091395:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04c3a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.091501:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493683584:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34047us (34449us total) trans 4294972807 rc 0/0 00000100:00100000:2.0:1789760723.091514:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5518 00000100:00100000:2.0:1789760723.091524:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733350211599 time 733398472824 state 0x2 ref 0 00000100:00100000:1.0:1789760723.095857:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760723.095943:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493683712 00000100:00100000:2.0:1789760723.095953:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493683712 00000100:00100000:2.0:1789760723.100238:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733350211599 time 733398472824 state 0x0 ref 1 00000100:00100000:2.0:1789760723.100259:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733398733240 time 733398472824 state 0x1 ref 1 00000100:00100000:2.0:1789760723.100295:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733398733240 time 733441331021 state 0x0 ref 1 00000100:00100000:2.0:1789760723.100309:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493683712:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760723.100358:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22482944, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb007 00010000:00010000:2.0:1789760723.100367:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04c41 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.100384:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760723.100388:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.100394:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04c41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.100407:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04c41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.129997:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972808 is committed 00002000:00010000:2.0:1789760723.130350:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22482944 -> 22487040 00010000:00010000:2.0:1789760723.130366:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04c41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.130379:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04c41 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.130388:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04c41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.130396:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760723.130402:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04c41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.130411:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04c41 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.130430:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04c41 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.130439:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04c41 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.130532:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493683712:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30221us (34678us total) trans 4294972808 rc 0/0 00000100:00100000:2.0:1789760723.130543:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5519 00000100:00100000:2.0:1789760723.130553:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733398733240 time 733441331021 state 0x2 ref 0 00000100:00100000:1.0:1789760723.133449:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760723.133507:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493683840 00000100:00100000:0.0:1789760723.133516:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493683840 00000100:00100000:0.0:1789760723.133568:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733398733240 time 733441331021 state 0x0 ref 1 00000100:00100000:0.0:1789760723.133583:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733441591437 time 733441331021 state 0x1 ref 1 00000100:00100000:2.0:1789760723.133648:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733441591437 time 733474680633 state 0x0 ref 1 00000100:00100000:2.0:1789760723.133664:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493683840:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760723.133729:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22487040, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb007 00010000:00010000:2.0:1789760723.133738:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04c48 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.133754:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760723.133758:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.133764:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04c48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.133778:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04c48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.160368:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972809 is committed 00002000:00010000:3.0:1789760723.160372:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22487040 -> 22491136 00010000:00010000:3.0:1789760723.160388:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04c48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.160401:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04c48 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.160411:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04c48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.160418:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760723.160423:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04c48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.160433:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04c48 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.160454:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04c48 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.160461:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e04c48 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760723.160559:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493683840:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26893us (27111us total) trans 4294972809 rc 0/0 00000100:00100000:3.0:1789760723.160570:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5520 00000100:00100000:3.0:1789760723.160580:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733441591437 time 733474680633 state 0x2 ref 0 00000100:00100000:0.0:1789760723.163663:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760723.164317:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493683968 00000100:00100000:3.0:1789760723.164327:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493683968 00000100:00100000:0.0:1789760723.164450:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733441591437 time 733474680633 state 0x0 ref 1 00000100:00100000:0.0:1789760723.164465:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733474941049 time 733474680633 state 0x1 ref 1 00000100:00100000:0.0:1789760723.164500:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733474941049 time 733505534717 state 0x0 ref 1 00000100:00100000:0.0:1789760723.164512:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493683968:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760723.164559:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22491136, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb007 00010000:00010000:0.0:1789760723.164567:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04c4f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.164583:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760723.164587:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.164593:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04c4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.164608:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04c4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.219876:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972810 is committed 00002000:00010000:0.0:1789760723.220010:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22491136 -> 22495232 00010000:00010000:0.0:1789760723.220026:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04c4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.220039:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04c4f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.220047:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04c4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.220054:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760723.220059:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04c4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.220070:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04c4f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.220090:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04c4f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.220098:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04c4f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.220253:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493683968:12345-192.168.203.60@tcp:4:dd.0 Request processed in 55736us (56593us total) trans 4294972810 rc 0/0 00000100:00100000:0.0:1789760723.220270:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5521 00000100:00100000:0.0:1789760723.220281:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733474941049 time 733505534717 state 0x2 ref 0 00000100:00100000:1.0:1789760723.226884:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760723.227435:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493684096 00000100:00100000:0.0:1789760723.227446:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493684096 00000100:00100000:0.0:1789760723.227501:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733474941049 time 733505534717 state 0x0 ref 1 00000100:00100000:0.0:1789760723.227516:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733505795133 time 733505534717 state 0x1 ref 1 00000100:00100000:0.0:1789760723.227597:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733505795133 time 733568631659 state 0x0 ref 1 00000100:00100000:0.0:1789760723.227630:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493684096:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760723.227681:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22495232, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb007 00010000:00010000:0.0:1789760723.227689:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04c56 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.227705:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760723.227709:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.227715:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04c56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.227729:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04c56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760723.259939:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972811 is committed 00002000:00010000:1.0:1789760723.260103:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22495232 -> 22499328 00010000:00010000:1.0:1789760723.260119:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04c56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.260132:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04c56 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.260140:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04c56 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.260147:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760723.260152:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04c56 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.260162:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04c56 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.260182:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04c56 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.260190:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04c56 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.260296:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493684096:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32664us (33416us total) trans 4294972811 rc 0/0 00000100:00100000:1.0:1789760723.260307:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5522 00000100:00100000:1.0:1789760723.260320:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733505795133 time 733568631659 state 0x2 ref 0 00000100:00100000:1.0:1789760723.268744:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760723.270155:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493684224 00000100:00100000:2.0:1789760723.270170:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493684224 00000100:00100000:2.0:1789760723.270240:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733505795133 time 733568631659 state 0x0 ref 1 00000100:00100000:2.0:1789760723.270260:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733568892075 time 733568631659 state 0x1 ref 1 00000100:00100000:2.0:1789760723.270401:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733568892075 time 733611374072 state 0x0 ref 1 00000100:00100000:2.0:1789760723.270417:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493684224:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760723.270469:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22499328, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb007 00010000:00010000:2.0:1789760723.270479:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04c5d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.270495:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760723.270500:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.270506:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04c5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.270520:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04c5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760723.301229:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972812 is committed 00002000:00010000:0.0:1789760723.301305:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22499328 -> 22503424 00010000:00010000:0.0:1789760723.301322:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04c5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.301441:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04c5d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.301448:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04c5d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.301453:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760723.301457:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04c5d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.301463:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04c5d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.301479:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04c5d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.301484:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04c5d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.301565:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493684224:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31147us (32826us total) trans 4294972812 rc 0/0 00000100:00100000:0.0:1789760723.301585:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5523 00000100:00100000:0.0:1789760723.301597:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733568892075 time 733611374072 state 0x2 ref 0 00000100:00100000:0.0:1789760723.305660:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760723.305794:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493684352 00000100:00100000:3.0:1789760723.305804:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493684352 00000100:00100000:3.0:1789760723.305860:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733568892075 time 733611374072 state 0x0 ref 1 00000100:00100000:3.0:1789760723.305875:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733611634488 time 733611374072 state 0x1 ref 1 00000100:00100000:3.0:1789760723.305912:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733611634488 time 733646947617 state 0x0 ref 1 00000100:00100000:3.0:1789760723.305924:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc17480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493684352:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760723.306001:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22503424, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb007 00010000:00010000:3.0:1789760723.306010:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04c64 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.306031:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760723.306035:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760723.306041:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04c64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.306053:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04c64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.322439:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc5837b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3303:1876699519249280:0@lo:400:kworker.0 00000100:00100000:1.0:1789760723.322504:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760723.323255:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519249280 00000100:00100000:2.0:1789760723.323266:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519249280 00000100:00100000:2.0:1789760723.323320:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 392 00000100:00100000:2.0:1789760723.323330:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d4c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3303:x1876699519249280:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760723.323588:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d4c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3303:x1876699519249280:12345-0@lo:400:kworker.0 Request processed in 259us (1082us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760723.323597:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 392 00000100:00100000:3.0:1789760723.329374:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc5834a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3304:1876699519249408:0@lo:400:kworker.0 00000100:00100000:3.0:1789760723.329430:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760723.329460:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760723.329465:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc5835c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3304:1876699519249792:0@lo:400:kworker.0 00000100:00100000:3.0:1789760723.329483:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760723.329501:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc5835880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3305:1876699519249536:0@lo:400:kworker.0 00000100:00100000:3.0:1789760723.329581:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760723.329610:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789760723.329628:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc5837800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3305:1876699519249664:0@lo:400:kworker.0 00000100:00100000:3.0:1789760723.329656:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760723.329696:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519249792 00000100:00100000:3.0:1789760723.329714:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519249792 00000100:00100000:3.0:1789760723.329779:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 169 00000100:00100000:3.0:1789760723.329791:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519249792:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760723.329845:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519249792:12345-0@lo:400:kworker.0 Request processed in 58us (359us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760723.329852:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 169 00000100:00100000:3.0:1789760723.329878:0:10519:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519249408 00000100:00100000:3.0:1789760723.329882:0:10519:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519249408 00000100:00100000:3.0:1789760723.329912:0:10519:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519249536 00000100:00100000:3.0:1789760723.329915:0:10519:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519249536 00000100:00100000:3.0:1789760723.329942:0:10519:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519249664 00000100:00100000:3.0:1789760723.329944:0:10519:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519249664 00000100:00100000:3.0:1789760723.329981:0:10519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 464 00000100:00100000:3.0:1789760723.329991:0:10519:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3304:x1876699519249408:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760723.330035:0:10519:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3304:x1876699519249408:12345-0@lo:400:kworker.0 Request processed in 47us (605us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760723.330042:0:10519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 464 00000100:00100000:3.0:1789760723.330055:0:10519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 465 00000100:00100000:3.0:1789760723.330062:0:10519:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc352c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3305:x1876699519249536:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760723.330100:0:10519:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3305:x1876699519249536:12345-0@lo:400:kworker.0 Request processed in 38us (520us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760723.330107:0:10519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 465 00000100:00100000:3.0:1789760723.330121:0:10519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 466 00000100:00100000:3.0:1789760723.330128:0:10519:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc352ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3305:x1876699519249664:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760723.330160:0:10519:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3305:x1876699519249664:12345-0@lo:400:kworker.0 Request processed in 34us (504us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760723.330166:0:10519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 466 00000100:00080000:3.0:1789760723.330188:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789760723.330193:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789760723.330236:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc5835c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3304:1876699519249792:0@lo:400:kworker.0 00000100:00100000:3.0:1789760723.330248:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc5834a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3304:1876699519249408:0@lo:400:kworker.0 00000100:00100000:3.0:1789760723.330279:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc5837800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3305:1876699519249664:0@lo:400:kworker.0 00000100:00100000:3.0:1789760723.330292:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc5835880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3305:1876699519249536:0@lo:400:kworker.0 00000100:00100000:1.0:1789760723.330815:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc5837b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3303:1876699519249280:0@lo:400:kworker.0 00000001:00080000:1.0:1789760723.341373:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972813 is committed 00002000:00010000:3.0:1789760723.341476:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22503424 -> 22507520 00010000:00010000:3.0:1789760723.341491:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04c64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.341504:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04c64 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.341512:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04c64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.341519:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760723.341523:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04c64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.341532:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04c64 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.341550:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04c64 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.341556:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04c64 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760723.341651:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc17480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493684352:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35725us (35996us total) trans 4294972813 rc 0/0 00000100:00100000:3.0:1789760723.342466:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5524 00000100:00100000:3.0:1789760723.342482:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733611634488 time 733646947617 state 0x2 ref 0 00000100:00100000:1.0:1789760723.344850:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760723.345004:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493684480 00000100:00100000:0.0:1789760723.345057:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493684480 00000100:00100000:2.0:1789760723.345230:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733611634488 time 733646947617 state 0x0 ref 1 00000100:00100000:2.0:1789760723.345243:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733647208033 time 733646947617 state 0x1 ref 1 00000100:00100000:2.0:1789760723.345266:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733647208033 time 733686302855 state 0x0 ref 1 00000100:00100000:2.0:1789760723.345275:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493684480:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760723.345311:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22507520, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb007 00010000:00010000:2.0:1789760723.345317:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04c6b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.345329:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760723.345334:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.345340:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04c6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.345354:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04c6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.371710:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972814 is committed 00002000:00010000:0.0:1789760723.371905:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22507520 -> 22511616 00010000:00010000:0.0:1789760723.371921:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04c6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.371935:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04c6b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.371944:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04c6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.371952:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760723.371957:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04c6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.371987:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04c6b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.372007:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04c6b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.372015:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04c6b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.372180:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493684480:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26899us (27331us total) trans 4294972814 rc 0/0 00000100:00100000:0.0:1789760723.372191:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5525 00000100:00100000:0.0:1789760723.372201:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733647208033 time 733686302855 state 0x2 ref 0 00000100:00100000:1.0:1789760723.375443:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760723.375516:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493684608 00000100:00100000:0.0:1789760723.375526:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493684608 00000100:00100000:0.0:1789760723.375574:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733647208033 time 733686302855 state 0x0 ref 1 00000100:00100000:0.0:1789760723.375589:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733686563271 time 733686302855 state 0x1 ref 1 00000100:00100000:0.0:1789760723.375625:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733686563271 time 733716661136 state 0x0 ref 1 00000100:00100000:0.0:1789760723.375638:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493684608:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760723.375686:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22511616, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb007 00010000:00010000:0.0:1789760723.375695:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04c72 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.375718:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760723.375722:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.375728:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04c72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.375742:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04c72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760723.429738:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972815 is committed 00002000:00010000:1.0:1789760723.429845:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22511616 -> 22515712 00010000:00010000:1.0:1789760723.429861:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04c72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.429875:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04c72 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.429883:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04c72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.429890:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760723.429895:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04c72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.429905:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04c72 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.429944:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04c72 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.429952:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04c72 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.430069:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493684608:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54427us (54629us total) trans 4294972815 rc 0/0 00000100:00100000:1.0:1789760723.430080:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5526 00000100:00100000:1.0:1789760723.430090:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733686563271 time 733716661136 state 0x2 ref 0 00000100:00100000:1.0:1789760723.435042:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760723.435155:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493684736 00000100:00100000:0.0:1789760723.435164:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493684736 00000100:00100000:0.0:1789760723.435223:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733686563271 time 733716661136 state 0x0 ref 1 00000100:00100000:0.0:1789760723.435238:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733716921552 time 733716661136 state 0x1 ref 1 00000100:00100000:0.0:1789760723.435271:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733716921552 time 733776307169 state 0x0 ref 1 00000100:00100000:0.0:1789760723.435283:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493684736:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760723.435332:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22515712, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb007 00010000:00010000:0.0:1789760723.435341:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04c79 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.435356:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760723.435360:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.435367:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04c79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.435381:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04c79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760723.469129:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972816 is committed 00002000:00010000:1.0:1789760723.469378:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22515712 -> 22519808 00010000:00010000:1.0:1789760723.469394:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04c79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.469407:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04c79 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.469417:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04c79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.469424:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760723.469429:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04c79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.469444:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04c79 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.469464:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04c79 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.469472:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04c79 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.469559:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493684736:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34273us (34522us total) trans 4294972816 rc 0/0 00000100:00100000:1.0:1789760723.469570:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5527 00000100:00100000:1.0:1789760723.469581:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733716921552 time 733776307169 state 0x2 ref 0 00000100:00100000:3.0:1789760723.474652:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760723.476624:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493684864 00000100:00100000:0.0:1789760723.476639:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493684864 00000100:00100000:0.0:1789760723.476703:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733716921552 time 733776307169 state 0x0 ref 1 00000100:00100000:0.0:1789760723.476719:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733776567585 time 733776307169 state 0x1 ref 1 00000100:00100000:0.0:1789760723.476752:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733776567585 time 733817788458 state 0x0 ref 1 00000100:00100000:0.0:1789760723.476771:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493684864:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760723.476827:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22519808, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb007 00010000:00010000:0.0:1789760723.476837:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04c80 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.476852:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760723.476859:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.476865:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04c80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.476881:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04c80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.514639:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760723.519720:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22519808 -> 22523904 00010000:00010000:0.0:1789760723.519740:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04c80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.519753:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04c80 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.519762:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04c80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.519770:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760723.519775:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04c80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.519785:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04c80 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.519806:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04c80 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.519814:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04c80 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.520334:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493684864:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43142us (45266us total) trans 4294972817 rc 0/0 00000100:00100000:0.0:1789760723.520417:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5528 00000100:00100000:0.0:1789760723.520426:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733776567585 time 733817788458 state 0x2 ref 0 00000001:00080000:1.0:1789760723.520549:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972817 is committed 00000100:00100000:0.0:1789760723.524191:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760723.524330:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493684992 00000100:00100000:2.0:1789760723.524464:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493684992 00000100:00100000:2.0:1789760723.524523:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733776567585 time 733817788458 state 0x0 ref 1 00000100:00100000:2.0:1789760723.524538:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733818048874 time 733817788458 state 0x1 ref 1 00000100:00100000:2.0:1789760723.524569:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733818048874 time 733865604556 state 0x0 ref 1 00000100:00100000:2.0:1789760723.524581:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493684992:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760723.524643:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22523904, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb007 00010000:00010000:2.0:1789760723.524652:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04c87 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.524668:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760723.524775:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.524783:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04c87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.524799:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04c87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.554627:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972818 is committed 00002000:00010000:2.0:1789760723.554801:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22523904 -> 22528000 00010000:00010000:2.0:1789760723.554819:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04c87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.554832:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04c87 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.554842:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04c87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.554850:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760723.554856:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04c87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.554866:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04c87 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.554886:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04c87 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.554895:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04c87 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.555033:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493684992:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30448us (30846us total) trans 4294972818 rc 0/0 00000100:00100000:2.0:1789760723.555048:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5529 00000100:00100000:2.0:1789760723.555058:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733818048874 time 733865604556 state 0x2 ref 0 00000100:00100000:2.0:1789760723.559162:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760723.559231:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493685120 00000100:00100000:1.0:1789760723.559242:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493685120 00000100:00100000:1.0:1789760723.559299:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733818048874 time 733865604556 state 0x0 ref 1 00000100:00100000:1.0:1789760723.559315:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733865864972 time 733865604556 state 0x1 ref 1 00000100:00100000:1.0:1789760723.559335:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733865864972 time 733900371647 state 0x0 ref 1 00000100:00100000:1.0:1789760723.559424:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493685120:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760723.559476:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22528000, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb007 00010000:00010000:1.0:1789760723.559485:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c8e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.559501:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760723.559505:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.559511:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.559524:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760723.586735:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972819 is committed 00002000:00010000:2.0:1789760723.586801:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22528000 -> 22532096 00010000:00010000:2.0:1789760723.586817:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.586831:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c8e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.586840:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.586847:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760723.586852:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.586862:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c8e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.586882:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c8e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.586890:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04c8e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.587044:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493685120:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27620us (27887us total) trans 4294972819 rc 0/0 00000100:00100000:2.0:1789760723.587055:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5530 00000100:00100000:2.0:1789760723.587064:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733865864972 time 733900371647 state 0x2 ref 0 00000100:00100000:0.0:1789760723.590884:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760723.591016:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493685248 00000100:00100000:1.0:1789760723.591027:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493685248 00000100:00100000:1.0:1789760723.591088:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733865864972 time 733900371647 state 0x0 ref 1 00000100:00100000:1.0:1789760723.591102:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733900632063 time 733900371647 state 0x1 ref 1 00000100:00100000:1.0:1789760723.591127:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733900632063 time 733932163513 state 0x0 ref 1 00000100:00100000:1.0:1789760723.591140:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493685248:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760723.591187:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22532096, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0xb007 00010000:00010000:1.0:1789760723.591195:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04c95 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.591210:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760723.591215:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.591221:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04c95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.591233:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04c95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760723.621919:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22532096 -> 22536192 00002000:00010000:1.0:1789760723.621929:0:13038:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760722 -> 1789760723 00002000:00010000:1.0:1789760723.621932:0:13038:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760722 -> 1789760723 00010000:00010000:1.0:1789760723.621945:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04c95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.621983:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04c95 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.621994:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04c95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.622002:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760723.622007:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04c95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.622017:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04c95 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.622038:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04c95 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.622046:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04c95 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760723.622274:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972820 is committed 00000100:00100000:1.0:1789760723.622327:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493685248:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31184us (31444us total) trans 4294972820 rc 0/0 00000100:00100000:1.0:1789760723.622338:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5531 00000100:00100000:1.0:1789760723.622350:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733900632063 time 733932163513 state 0x2 ref 0 00000100:00100000:0.0:1789760723.627668:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760723.627872:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493685376 00000100:00100000:3.0:1789760723.627882:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493685376 00000100:00100000:3.0:1789760723.627933:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733900632063 time 733932163513 state 0x0 ref 1 00000100:00100000:3.0:1789760723.627947:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733932423929 time 733932163513 state 0x1 ref 1 00000100:00100000:3.0:1789760723.628069:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733932423929 time 733969103133 state 0x0 ref 1 00000100:00100000:3.0:1789760723.628084:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493685376:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760723.628134:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22536192, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb007 00010000:00010000:3.0:1789760723.628143:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04c9c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.628159:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760723.628164:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760723.628170:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04c9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760723.628183:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04c9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.685435:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972821 is committed 00002000:00010000:2.0:1789760723.685685:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22536192 -> 22540288 00010000:00010000:2.0:1789760723.685714:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04c9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.685727:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04c9c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.685737:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04c9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.685744:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760723.685749:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04c9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.685758:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04c9c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.685782:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04c9c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.685796:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04c9c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.685880:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493685376:12345-192.168.203.60@tcp:4:dd.0 Request processed in 57795us (58227us total) trans 4294972821 rc 0/0 00000100:00100000:2.0:1789760723.685888:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5532 00000100:00100000:2.0:1789760723.685897:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733932423929 time 733969103133 state 0x2 ref 0 00000100:00100000:2.0:1789760723.690264:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760723.690418:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493685504 00000100:00100000:0.0:1789760723.690428:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493685504 00000100:00100000:0.0:1789760723.690635:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733932423929 time 733969103133 state 0x0 ref 1 00000100:00100000:0.0:1789760723.690727:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733969363549 time 733969103133 state 0x1 ref 1 00000100:00100000:0.0:1789760723.690759:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733969363549 time 734031795035 state 0x0 ref 1 00000100:00100000:0.0:1789760723.690774:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493685504:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760723.690824:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22540288, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb007 00010000:00010000:0.0:1789760723.690833:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04ca3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.690850:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760723.690855:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.690862:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04ca3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.690877:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04ca3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.730793:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972822 is committed 00002000:00010000:2.0:1789760723.731325:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22540288 -> 22544384 00010000:00010000:2.0:1789760723.731345:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04ca3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.731359:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04ca3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.731368:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04ca3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.731376:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760723.731382:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04ca3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.731392:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04ca3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.731421:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04ca3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.731430:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e04ca3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.731773:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493685504:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40993us (41510us total) trans 4294972822 rc 0/0 00000100:00100000:2.0:1789760723.731786:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5533 00000100:00100000:2.0:1789760723.731798:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733969363549 time 734031795035 state 0x2 ref 0 00000100:00100000:0.0:1789760723.735111:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760723.735257:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493685632 00000100:00100000:2.0:1789760723.735271:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493685632 00000100:00100000:2.0:1789760723.735341:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 733969363549 time 734031795035 state 0x0 ref 1 00000100:00100000:2.0:1789760723.735358:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734032055451 time 734031795035 state 0x1 ref 1 00000100:00100000:2.0:1789760723.735395:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734032055451 time 734076430286 state 0x0 ref 1 00000100:00100000:2.0:1789760723.735409:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493685632:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760723.735487:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22544384, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb007 00010000:00010000:2.0:1789760723.735498:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04caa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.735516:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760723.735692:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.735700:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04caa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.735717:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04caa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760723.784537:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972823 is committed 00002000:00010000:1.0:1789760723.784700:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22544384 -> 22548480 00010000:00010000:1.0:1789760723.784720:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04caa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.784736:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04caa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.784748:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04caa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.784756:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760723.784761:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04caa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.784772:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04caa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.784807:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04caa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.784816:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e04caa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.785045:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493685632:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49631us (49941us total) trans 4294972823 rc 0/0 00000100:00100000:1.0:1789760723.785059:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5534 00000100:00100000:1.0:1789760723.785072:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734032055451 time 734076430286 state 0x2 ref 0 00000100:00100000:0.0:1789760723.789930:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760723.790247:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493685760 00000100:00100000:1.0:1789760723.790260:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493685760 00000100:00100000:1.0:1789760723.790335:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734032055451 time 734076430286 state 0x0 ref 1 00000100:00100000:1.0:1789760723.790355:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734076690702 time 734076430286 state 0x1 ref 1 00000100:00100000:1.0:1789760723.790389:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734076690702 time 734131423461 state 0x0 ref 1 00000100:00100000:1.0:1789760723.790409:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493685760:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760723.790473:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22548480, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb007 00010000:00010000:1.0:1789760723.790483:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04cb1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.790505:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760723.790511:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.790520:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04cb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.790536:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04cb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760723.835001:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc5835880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3304:1876699519249920:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760723.835094:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760723.835186:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519249920 00000100:00100000:3.0:1789760723.835196:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519249920 00000100:00100000:3.0:1789760723.835252:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 193 00000100:00100000:3.0:1789760723.835262:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519249920:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760723.835306:0:6690:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760723.836956:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519249920:12345-0@lo:13:osp-pre-1-0.0 Request processed in 1686us (1852us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760723.836986:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 193 00000100:00100000:2.0:1789760723.837181:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc5835880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3304:1876699519249920:0@lo:13:osp-pre-1-0.0 00000001:00080000:0.0:1789760723.856668:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972824 is committed 00002000:00010000:1.0:1789760723.857022:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22548480 -> 22552576 00010000:00010000:1.0:1789760723.857042:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04cb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.857057:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04cb1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.857067:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04cb1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.857075:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760723.857080:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04cb1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.857468:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04cb1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.857496:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04cb1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.857511:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04cb1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.857648:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493685760:12345-192.168.203.60@tcp:4:dd.0 Request processed in 67240us (67727us total) trans 4294972824 rc 0/0 00000100:00100000:1.0:1789760723.857661:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5535 00000100:00100000:1.0:1789760723.857673:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734076690702 time 734131423461 state 0x2 ref 0 00000100:00100000:0.0:1789760723.862118:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760723.862455:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493685888 00000100:00100000:2.0:1789760723.862469:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493685888 00000100:00100000:2.0:1789760723.862552:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734076690702 time 734131423461 state 0x0 ref 1 00000100:00100000:2.0:1789760723.862572:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734131683877 time 734131423461 state 0x1 ref 1 00000100:00100000:2.0:1789760723.862618:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734131683877 time 734203644619 state 0x0 ref 1 00000100:00100000:2.0:1789760723.862633:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493685888:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760723.862710:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22552576, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb007 00010000:00010000:2.0:1789760723.862721:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e04cb8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.862742:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760723.862747:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.862755:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e04cb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.862771:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e04cb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.917170:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972825 is committed 00002000:00010000:0.0:1789760723.917380:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22552576 -> 22556672 00010000:00010000:0.0:1789760723.917401:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e04cb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.917417:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e04cb8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.917428:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e04cb8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.917449:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760723.917456:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e04cb8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.917468:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e04cb8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.917495:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e04cb8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.917504:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e04cb8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.917802:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493685888:12345-192.168.203.60@tcp:4:dd.0 Request processed in 55163us (55691us total) trans 4294972825 rc 0/0 00000100:00100000:0.0:1789760723.917814:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5536 00000100:00100000:0.0:1789760723.917829:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734131683877 time 734203644619 state 0x2 ref 0 00000100:00100000:0.0:1789760723.921936:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760723.922100:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493686016 00000100:00100000:2.0:1789760723.922116:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493686016 00000100:00100000:2.0:1789760723.922196:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734131683877 time 734203644619 state 0x0 ref 1 00000100:00100000:2.0:1789760723.922215:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734203905035 time 734203644619 state 0x1 ref 1 00000100:00100000:2.0:1789760723.922243:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734203905035 time 734263278311 state 0x0 ref 1 00000100:00100000:2.0:1789760723.922257:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493686016:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760723.922338:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22556672, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb007 00010000:00010000:2.0:1789760723.922349:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04cbf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.922369:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760723.922375:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.922383:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04cbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.922400:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04cbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.987266:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972826 is committed 00002000:00010000:1.0:1789760723.987354:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22556672 -> 22560768 00010000:00010000:1.0:1789760723.987374:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04cbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.987389:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04cbf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.987398:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04cbf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.987405:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760723.987410:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04cbf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.987421:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04cbf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.988102:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04cbf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.988116:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04cbf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.988242:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493686016:12345-192.168.203.60@tcp:4:dd.0 Request processed in 65981us (66316us total) trans 4294972826 rc 0/0 00000100:00100000:1.0:1789760723.988255:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5537 00000100:00100000:1.0:1789760723.988269:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734203905035 time 734263278311 state 0x2 ref 0 00000100:00100000:1.0:1789760723.991372:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760723.993810:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493686144 00000100:00100000:2.0:1789760723.993834:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493686144 00000100:00100000:2.0:1789760723.993932:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734203905035 time 734263278311 state 0x0 ref 1 00000100:00100000:2.0:1789760723.993953:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734263538727 time 734263278311 state 0x1 ref 1 00000100:00100000:2.0:1789760723.994037:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734263538727 time 734335071115 state 0x0 ref 1 00000100:00100000:2.0:1789760723.994054:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493686144:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760723.994763:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22560768, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb007 00010000:00010000:2.0:1789760723.994776:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cc6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.994796:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760723.994802:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.994810:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.994826:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760724.087775:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972827 is committed 00002000:00010000:2.0:1789760724.087888:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22560768 -> 22564864 00010000:00010000:2.0:1789760724.087909:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.087925:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cc6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.087936:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cc6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.087945:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760724.087951:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cc6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.088237:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cc6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.088262:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cc6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.088270:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cc6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.088417:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493686144:12345-192.168.203.60@tcp:4:dd.0 Request processed in 94362us (97056us total) trans 4294972827 rc 0/0 00000100:00100000:2.0:1789760724.088430:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5538 00000100:00100000:2.0:1789760724.088449:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734263538727 time 734335071115 state 0x2 ref 0 00000100:00100000:0.0:1789760724.096075:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760724.096298:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493686272 00000100:00100000:2.0:1789760724.096310:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493686272 00000100:00100000:2.0:1789760724.096383:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734263538727 time 734335071115 state 0x0 ref 1 00000100:00100000:2.0:1789760724.096402:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734335331531 time 734335071115 state 0x1 ref 1 00000100:00100000:2.0:1789760724.096439:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734335331531 time 734437474386 state 0x0 ref 1 00000100:00100000:2.0:1789760724.096455:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493686272:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760724.096520:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22564864, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb007 00010000:00010000:2.0:1789760724.096531:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04ccd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.096550:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760724.096556:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.096575:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04ccd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.096592:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04ccd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760724.159368:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22564864 -> 22568960 00010000:00010000:2.0:1789760724.159402:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04ccd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.159419:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04ccd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.159430:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04ccd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.159451:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760724.159456:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04ccd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.159467:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04ccd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.159501:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04ccd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.159509:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04ccd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.159669:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493686272:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63209us (63708us total) trans 4294972828 rc 0/0 00000100:00100000:2.0:1789760724.159681:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5539 00000100:00100000:2.0:1789760724.159694:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734335331531 time 734437474386 state 0x2 ref 0 00000001:00080000:3.0:1789760724.161438:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972828 is committed 00000100:00100000:0.0:1789760724.179875:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760724.180056:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493686400 00000100:00100000:2.0:1789760724.180070:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493686400 00000100:00100000:2.0:1789760724.180146:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734335331531 time 734437474386 state 0x0 ref 1 00000100:00100000:2.0:1789760724.180164:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734437734802 time 734437474386 state 0x1 ref 1 00000100:00100000:2.0:1789760724.180203:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734437734802 time 734521238490 state 0x0 ref 1 00000100:00100000:2.0:1789760724.180217:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493686400:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760724.180283:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22568960, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb007 00010000:00010000:2.0:1789760724.180294:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cd4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.180313:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760724.180320:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.180327:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.180343:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760724.310572:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972829 is committed 00002000:00010000:2.0:1789760724.313575:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22568960 -> 22573056 00010000:00010000:2.0:1789760724.313603:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.313629:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cd4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.313639:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cd4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.313647:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760724.313653:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cd4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.313833:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cd4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.313863:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cd4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.313871:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cd4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.314051:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493686400:12345-192.168.203.60@tcp:4:dd.0 Request processed in 133829us (134186us total) trans 4294972829 rc 0/0 00000100:00100000:2.0:1789760724.314064:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5540 00000100:00100000:2.0:1789760724.314077:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734437734802 time 734521238490 state 0x2 ref 0 00000100:00100000:1.0:1789760724.318429:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760724.318701:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493686528 00000100:00100000:2.0:1789760724.318714:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493686528 00000100:00100000:2.0:1789760724.318793:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734437734802 time 734521238490 state 0x0 ref 1 00000100:00100000:2.0:1789760724.318811:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734521498906 time 734521238490 state 0x1 ref 1 00000100:00100000:2.0:1789760724.318847:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734521498906 time 734659882459 state 0x0 ref 1 00000100:00100000:2.0:1789760724.318861:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493686528:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760724.318929:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22573056, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb007 00010000:00010000:2.0:1789760724.318940:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cdb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.318984:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760724.318990:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.318998:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cdb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.319015:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cdb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760724.394405:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972830 is committed 00002000:00010000:1.0:1789760724.394580:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22573056 -> 22577152 00010000:00010000:1.0:1789760724.394600:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cdb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.394625:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cdb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.394635:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cdb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.394642:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760724.394648:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cdb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.394667:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cdb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.394693:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cdb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.394702:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04cdb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760724.394830:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493686528:12345-192.168.203.60@tcp:4:dd.0 Request processed in 75963us (76413us total) trans 4294972830 rc 0/0 00000100:00100000:1.0:1789760724.394843:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5541 00000100:00100000:1.0:1789760724.394857:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734521498906 time 734659882459 state 0x2 ref 0 00000100:00100000:1.0:1789760724.399201:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760724.399366:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493686656 00000100:00100000:3.0:1789760724.399379:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493686656 00000100:00100000:3.0:1789760724.399458:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734521498906 time 734659882459 state 0x0 ref 1 00000100:00100000:3.0:1789760724.399477:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734660142875 time 734659882459 state 0x1 ref 1 00000100:00100000:3.0:1789760724.399512:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734660142875 time 734740547020 state 0x0 ref 1 00000100:00100000:3.0:1789760724.399525:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493686656:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760724.399598:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22577152, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb007 00010000:00010000:3.0:1789760724.399619:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04ce2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.399640:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760724.399645:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760724.399652:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04ce2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.399680:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04ce2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760724.462993:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972831 is committed 00002000:00010000:3.0:1789760724.463127:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22577152 -> 22581248 00010000:00010000:3.0:1789760724.463147:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04ce2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.463161:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04ce2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.463171:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04ce2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.463180:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760724.463186:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04ce2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.463197:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04ce2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.463221:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04ce2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.463231:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04ce2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.463354:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493686656:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63823us (64166us total) trans 4294972831 rc 0/0 00000100:00100000:3.0:1789760724.463367:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5542 00000100:00100000:3.0:1789760724.463379:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734660142875 time 734740547020 state 0x2 ref 0 00000100:00100000:3.0:1789760724.471489:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760724.473629:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493686784 00000100:00100000:1.0:1789760724.473650:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493686784 00000100:00100000:1.0:1789760724.473755:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734660142875 time 734740547020 state 0x0 ref 1 00000100:00100000:1.0:1789760724.473774:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734740807436 time 734740547020 state 0x1 ref 1 00000100:00100000:1.0:1789760724.473865:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734740807436 time 734814899730 state 0x0 ref 1 00000100:00100000:1.0:1789760724.473880:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493686784:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760724.473947:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22581248, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb007 00010000:00010000:1.0:1789760724.474010:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04ce9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.474044:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760724.474049:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760724.474057:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04ce9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.474074:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04ce9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760724.527192:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22581248 -> 22585344 00010000:00010000:1.0:1789760724.527220:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04ce9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.527237:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04ce9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.527246:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04ce9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.527254:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760724.527259:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04ce9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.527268:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04ce9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.527298:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04ce9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.527306:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04ce9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760724.527436:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493686784:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53551us (55954us total) trans 4294972832 rc 0/0 00000100:00100000:1.0:1789760724.527449:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5543 00000100:00100000:1.0:1789760724.527462:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734740807436 time 734814899730 state 0x2 ref 0 00000001:00080000:0.0:1789760724.532012:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972832 is committed 00000100:00100000:1.0:1789760724.534353:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760724.534441:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493686912 00000100:00100000:0.0:1789760724.534453:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493686912 00000100:00100000:0.0:1789760724.534529:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734740807436 time 734814899730 state 0x0 ref 1 00000100:00100000:0.0:1789760724.534549:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734815160146 time 734814899730 state 0x1 ref 1 00000100:00100000:0.0:1789760724.534581:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734815160146 time 734875616256 state 0x0 ref 1 00000100:00100000:0.0:1789760724.534595:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493686912:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760724.534785:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22585344, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb007 00010000:00010000:0.0:1789760724.534797:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04cf0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.534817:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760724.534823:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760724.534829:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04cf0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.534845:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04cf0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.540231:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760724.617471:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22585344 -> 22589440 00010000:00010000:0.0:1789760724.617498:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04cf0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.617514:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04cf0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.617523:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04cf0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.617536:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760724.617542:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04cf0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.617562:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04cf0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.617585:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04cf0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.617593:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04cf0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760724.617733:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493686912:12345-192.168.203.60@tcp:4:dd.0 Request processed in 83133us (83387us total) trans 4294972833 rc 0/0 00000100:00100000:0.0:1789760724.617745:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5544 00000100:00100000:0.0:1789760724.617758:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734815160146 time 734875616256 state 0x2 ref 0 00000001:00080000:1.0:1789760724.619158:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972833 is committed 00000100:00100000:3.0:1789760724.621179:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760724.621423:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493687040 00000100:00100000:0.0:1789760724.621437:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493687040 00000100:00100000:0.0:1789760724.621516:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734815160146 time 734875616256 state 0x0 ref 1 00000100:00100000:0.0:1789760724.621535:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734875876672 time 734875616256 state 0x1 ref 1 00000100:00100000:0.0:1789760724.621580:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734875876672 time 734962615104 state 0x0 ref 1 00000100:00100000:0.0:1789760724.621598:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493687040:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760724.621685:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22589440, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0xb007 00010000:00010000:0.0:1789760724.621696:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04cf7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.621716:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760724.621722:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760724.621730:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04cf7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.621747:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04cf7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760724.655351:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22589440 -> 22593536 00002000:00010000:0.0:1789760724.655363:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760723 -> 1789760724 00002000:00010000:0.0:1789760724.655366:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760723 -> 1789760724 00010000:00010000:0.0:1789760724.655384:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04cf7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.655401:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04cf7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.655411:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04cf7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.655420:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760724.655426:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04cf7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.655436:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04cf7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.655461:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04cf7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.655470:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e04cf7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760724.655602:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493687040:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33999us (34432us total) trans 4294972834 rc 0/0 00000100:00100000:0.0:1789760724.655623:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5545 00000100:00100000:0.0:1789760724.655634:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734875876672 time 734962615104 state 0x2 ref 0 00000001:00080000:1.0:1789760724.658916:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972834 is committed 00000100:00100000:3.0:1789760724.660327:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760724.660429:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493687168 00000100:00100000:1.0:1789760724.660441:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493687168 00000100:00100000:1.0:1789760724.660508:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734875876672 time 734962615104 state 0x0 ref 1 00000100:00100000:1.0:1789760724.660528:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734962875520 time 734962615104 state 0x1 ref 1 00000100:00100000:0.0:1789760724.660651:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734962875520 time 735001682999 state 0x0 ref 1 00000100:00100000:0.0:1789760724.660678:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493687168:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760724.660746:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22593536, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:0.0:1789760724.660757:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04cfe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.660777:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760724.660783:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760724.660792:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04cfe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.660812:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04cfe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760724.705816:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972835 is committed 00002000:00010000:1.0:1789760724.710984:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22593536 -> 22597632 00010000:00010000:1.0:1789760724.711011:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04cfe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.711027:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04cfe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.711043:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04cfe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.711051:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760724.711056:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04cfe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.711066:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04cfe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.711224:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04cfe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.711235:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04cfe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760724.711380:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493687168:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50699us (51054us total) trans 4294972835 rc 0/0 00000100:00100000:1.0:1789760724.711393:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5546 00000100:00100000:1.0:1789760724.711406:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734962875520 time 735001682999 state 0x2 ref 0 00000100:00100000:3.0:1789760724.716184:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760724.716447:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493687296 00000100:00100000:1.0:1789760724.716460:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493687296 00000100:00100000:1.0:1789760724.716536:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 734962875520 time 735001682999 state 0x0 ref 1 00000100:00100000:1.0:1789760724.716557:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735001943415 time 735001682999 state 0x1 ref 1 00000100:00100000:1.0:1789760724.716584:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735001943415 time 735057619284 state 0x0 ref 1 00000100:00100000:1.0:1789760724.716598:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493687296:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760724.716676:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22597632, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:1.0:1789760724.716698:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04d05 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.716718:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760724.716723:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760724.716730:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04d05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.716747:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04d05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760724.792208:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972836 is committed 00002000:00010000:3.0:1789760724.792322:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22597632 -> 22601728 00010000:00010000:3.0:1789760724.792342:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04d05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.792357:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04d05 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.792366:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04d05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.792374:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760724.792381:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04d05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.792398:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04d05 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.792518:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04d05 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.792526:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04d05 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.792644:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493687296:12345-192.168.203.60@tcp:4:dd.0 Request processed in 76042us (76468us total) trans 4294972836 rc 0/0 00000100:00100000:3.0:1789760724.792656:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5547 00000100:00100000:3.0:1789760724.792731:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735001943415 time 735057619284 state 0x2 ref 0 00000100:00100000:3.0:1789760724.802313:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760724.802456:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493687424 00000100:00100000:0.0:1789760724.802469:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493687424 00000100:00100000:0.0:1789760724.802542:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735001943415 time 735057619284 state 0x0 ref 1 00000100:00100000:0.0:1789760724.802563:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735057879700 time 735057619284 state 0x1 ref 1 00000100:00100000:0.0:1789760724.802611:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735057879700 time 735143635540 state 0x0 ref 1 00000100:00100000:0.0:1789760724.802625:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493687424:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760724.802768:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22601728, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:1.0:1789760724.802781:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04d0c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.802809:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760724.802814:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760724.802821:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04d0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.802839:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04d0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760724.857113:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22601728 -> 22605824 00010000:00010000:3.0:1789760724.857139:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04d0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.857153:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04d0c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.857162:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04d0c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.857175:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760724.857181:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04d0c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.857190:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04d0c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.857214:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04d0c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.857222:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04d0c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.857345:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493687424:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54714us (55040us total) trans 4294972837 rc 0/0 00000100:00100000:3.0:1789760724.857357:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5548 00000100:00100000:3.0:1789760724.857370:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735057879700 time 735143635540 state 0x2 ref 0 00000001:00080000:0.0:1789760724.857927:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972837 is committed 00000100:00100000:0.0:1789760724.866346:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760724.866481:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493687552 00000100:00100000:2.0:1789760724.866498:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493687552 00000100:00100000:2.0:1789760724.866588:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735057879700 time 735143635540 state 0x0 ref 1 00000100:00100000:2.0:1789760724.866617:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735143895956 time 735143635540 state 0x1 ref 1 00000100:00100000:2.0:1789760724.866736:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735143895956 time 735207770305 state 0x0 ref 1 00000100:00100000:2.0:1789760724.866752:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493687552:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760724.866824:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22605824, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:2.0:1789760724.866937:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04d13 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.866958:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760724.866983:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.866991:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04d13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.867008:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04d13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760724.909030:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972838 is committed 00002000:00010000:0.0:1789760724.909306:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22605824 -> 22609920 00010000:00010000:0.0:1789760724.909327:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04d13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.909342:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04d13 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.909352:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04d13 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.909361:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760724.909367:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04d13 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.909377:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04d13 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.909403:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04d13 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.909411:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04d13 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760724.909581:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493687552:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42824us (43244us total) trans 4294972838 rc 0/0 00000100:00100000:0.0:1789760724.909594:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5549 00000100:00100000:0.0:1789760724.909614:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735143895956 time 735207770305 state 0x2 ref 0 00000100:00100000:2.0:1789760724.913537:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760724.913623:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493687680 00000100:00100000:0.0:1789760724.913637:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493687680 00000100:00100000:0.0:1789760724.913718:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735143895956 time 735207770305 state 0x0 ref 1 00000100:00100000:0.0:1789760724.913736:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735208030721 time 735207770305 state 0x1 ref 1 00000100:00100000:0.0:1789760724.913770:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735208030721 time 735254805364 state 0x0 ref 1 00000100:00100000:0.0:1789760724.913789:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493687680:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760724.913874:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22609920, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:0.0:1789760724.913884:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04d1a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.913903:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760724.913908:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760724.913915:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04d1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.913933:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04d1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760724.958259:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972839 is committed 00002000:00010000:1.0:1789760724.958448:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22609920 -> 22614016 00010000:00010000:1.0:1789760724.958466:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04d1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.958477:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04d1a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.958484:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04d1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.958488:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760724.958492:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04d1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.958501:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04d1a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.958523:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04d1a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.958530:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e04d1a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760724.958816:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493687680:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45023us (45288us total) trans 4294972839 rc 0/0 00000100:00100000:1.0:1789760724.958842:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5550 00000100:00100000:1.0:1789760724.958856:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735208030721 time 735254805364 state 0x2 ref 0 00000100:00100000:1.0:1789760724.963324:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760724.963416:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493687808 00000100:00100000:2.0:1789760724.963431:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493687808 00000100:00100000:2.0:1789760724.963511:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735208030721 time 735254805364 state 0x0 ref 1 00000100:00100000:2.0:1789760724.963530:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735255065780 time 735254805364 state 0x1 ref 1 00000100:00100000:2.0:1789760724.963561:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735255065780 time 735304597234 state 0x0 ref 1 00000100:00100000:2.0:1789760724.963576:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493687808:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760724.964196:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22614016, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:2.0:1789760724.964212:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04d21 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.964234:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760724.964250:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.964259:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04d21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.964278:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04d21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760725.016879:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22614016 -> 22618112 00010000:00010000:2.0:1789760725.016910:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04d21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.016927:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04d21 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.016943:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04d21 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.016951:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760725.016957:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04d21 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.017009:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04d21 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.017033:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04d21 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.017041:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04d21 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.017160:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493687808:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53579us (53844us total) trans 4294972840 rc 0/0 00000100:00100000:2.0:1789760725.017172:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5551 00000100:00100000:2.0:1789760725.017184:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735255065780 time 735304597234 state 0x2 ref 0 00000001:00080000:1.0:1789760725.017942:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972840 is committed 00000100:00100000:1.0:1789760725.022323:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760725.022499:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493687936 00000100:00100000:3.0:1789760725.022515:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493687936 00000100:00080000:0.0:1789760725.022578:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760725.022618:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735255065780 time 735304597234 state 0x0 ref 1 00000100:00100000:3.0:1789760725.022637:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735304857650 time 735304597234 state 0x1 ref 1 00000100:00100000:3.0:1789760725.022783:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735304857650 time 735363816130 state 0x0 ref 1 00000100:00100000:3.0:1789760725.022800:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493687936:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760725.022879:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22618112, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:3.0:1789760725.022895:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04d28 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.022917:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760725.022924:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.022934:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04d28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.022952:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04d28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760725.083591:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972841 is committed 00002000:00010000:0.0:1789760725.083735:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22618112 -> 22622208 00010000:00010000:0.0:1789760725.083755:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04d28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.083770:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04d28 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.083780:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04d28 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.083788:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760725.083793:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04d28 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.083804:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04d28 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.083837:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04d28 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.083847:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e04d28 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760725.084564:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493687936:12345-192.168.203.60@tcp:4:dd.0 Request processed in 61759us (62249us total) trans 4294972841 rc 0/0 00000100:00100000:0.0:1789760725.084579:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5552 00000100:00100000:0.0:1789760725.084592:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735304857650 time 735363816130 state 0x2 ref 0 00000100:00100000:0.0:1789760725.089221:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760725.089322:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493688064 00000100:00100000:2.0:1789760725.089335:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493688064 00000100:00100000:2.0:1789760725.089413:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735304857650 time 735363816130 state 0x0 ref 1 00000100:00100000:2.0:1789760725.089432:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735364076546 time 735363816130 state 0x1 ref 1 00000100:00100000:2.0:1789760725.089510:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735364076546 time 735430544232 state 0x0 ref 1 00000100:00100000:2.0:1789760725.089525:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493688064:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760725.089594:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22622208, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:2.0:1789760725.089613:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04d2f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.089633:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760725.089638:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.089645:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04d2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.089728:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04d2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760725.132809:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22622208 -> 22626304 00010000:00010000:1.0:1789760725.132844:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04d2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.132860:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04d2f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.132871:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04d2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.132879:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760725.132885:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04d2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.132897:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04d2f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.132922:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04d2f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.132939:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04d2f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.133259:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493688064:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43729us (44046us total) trans 4294972842 rc 0/0 00000100:00100000:1.0:1789760725.133272:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5553 00000100:00100000:1.0:1789760725.133285:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735364076546 time 735430544232 state 0x2 ref 0 00000001:00080000:1.0:1789760725.133339:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972842 is committed 00000100:00100000:1.0:1789760725.136897:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760725.137165:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493688192 00000100:00100000:0.0:1789760725.137179:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493688192 00000100:00100000:0.0:1789760725.137252:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735364076546 time 735430544232 state 0x0 ref 1 00000100:00100000:0.0:1789760725.137271:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735430804648 time 735430544232 state 0x1 ref 1 00000100:00100000:0.0:1789760725.137345:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735430804648 time 735478380467 state 0x0 ref 1 00000100:00100000:0.0:1789760725.137360:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493688192:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760725.137422:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22626304, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:0.0:1789760725.137432:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04d36 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.137451:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760725.137455:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760725.137462:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04d36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.137479:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04d36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760725.202565:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22626304 -> 22630400 00010000:00010000:1.0:1789760725.202593:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04d36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.202618:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04d36 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.202630:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04d36 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.202639:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760725.202645:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04d36 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.202655:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04d36 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.202680:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04d36 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.202690:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e04d36 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.202811:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493688192:12345-192.168.203.60@tcp:4:dd.0 Request processed in 65446us (65923us total) trans 4294972843 rc 0/0 00000100:00100000:1.0:1789760725.202824:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5554 00000100:00100000:1.0:1789760725.202836:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735430804648 time 735478380467 state 0x2 ref 0 00000001:00080000:1.0:1789760725.202895:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972843 is committed 00000100:00100000:1.0:1789760725.215886:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760725.216128:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493688320 00000100:00100000:3.0:1789760725.216142:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493688320 00000100:00100000:3.0:1789760725.216222:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735430804648 time 735478380467 state 0x0 ref 1 00000100:00100000:3.0:1789760725.216240:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735478640883 time 735478380467 state 0x1 ref 1 00000100:00100000:3.0:1789760725.216291:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735478640883 time 735557324994 state 0x0 ref 1 00000100:00100000:3.0:1789760725.216307:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb06c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493688320:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760725.216376:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22630400, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:3.0:1789760725.216387:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04d3d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.216408:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760725.216414:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.216423:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04d3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.216440:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04d3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760725.276396:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972844 is committed 00002000:00010000:3.0:1789760725.276575:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22630400 -> 22634496 00010000:00010000:3.0:1789760725.276593:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04d3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.276619:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04d3d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.276630:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04d3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.276638:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760725.276643:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04d3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.276662:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04d3d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.276693:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04d3d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.276701:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04d3d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.276820:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb06c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493688320:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60509us (60943us total) trans 4294972844 rc 0/0 00000100:00100000:3.0:1789760725.276832:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5555 00000100:00100000:3.0:1789760725.276845:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735478640883 time 735557324994 state 0x2 ref 0 00000100:00100000:0.0:1789760725.285063:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760725.285165:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493688448 00000100:00100000:3.0:1789760725.285180:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493688448 00000100:00100000:3.0:1789760725.285259:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735478640883 time 735557324994 state 0x0 ref 1 00000100:00100000:3.0:1789760725.285279:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735557585410 time 735557324994 state 0x1 ref 1 00000100:00100000:3.0:1789760725.285317:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735557585410 time 735626352219 state 0x0 ref 1 00000100:00100000:3.0:1789760725.285331:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493688448:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760725.285399:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22634496, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:3.0:1789760725.285410:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04d44 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.285430:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760725.285436:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.285443:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04d44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.285460:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04d44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760725.333175:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972845 is committed 00002000:00010000:0.0:1789760725.334628:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22634496 -> 22638592 00010000:00010000:0.0:1789760725.334653:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04d44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.334668:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04d44 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.334677:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04d44 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.334685:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760725.334691:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04d44 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.334701:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04d44 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.334728:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04d44 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.334738:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04d44 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760725.334873:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493688448:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49536us (49818us total) trans 4294972845 rc 0/0 00000100:00100000:0.0:1789760725.334885:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5556 00000100:00100000:0.0:1789760725.334899:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735557585410 time 735626352219 state 0x2 ref 0 00000100:00100000:1.0:1789760725.339067:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760725.339278:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493688576 00000100:00100000:2.0:1789760725.339297:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493688576 00000100:00100000:2.0:1789760725.339379:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735557585410 time 735626352219 state 0x0 ref 1 00000100:00100000:2.0:1789760725.339399:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735626612635 time 735626352219 state 0x1 ref 1 00000100:00100000:2.0:1789760725.339435:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735626612635 time 735680470248 state 0x0 ref 1 00000100:00100000:2.0:1789760725.339450:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493688576:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760725.339516:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22638592, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:2.0:1789760725.339648:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04d4b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.339729:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760725.339734:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.339742:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04d4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.339760:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04d4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760725.396004:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972846 is committed 00002000:00010000:2.0:1789760725.396237:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22638592 -> 22642688 00010000:00010000:2.0:1789760725.396257:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04d4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.396271:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04d4b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.396281:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04d4b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.396290:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760725.396295:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04d4b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.396305:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04d4b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.396334:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04d4b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.396341:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04d4b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.396509:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493688576:12345-192.168.203.60@tcp:4:dd.0 Request processed in 57053us (57451us total) trans 4294972846 rc 0/0 00000100:00100000:2.0:1789760725.396533:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5557 00000100:00100000:2.0:1789760725.396547:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735626612635 time 735680470248 state 0x2 ref 0 00000100:00100000:1.0:1789760725.399419:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760725.399569:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493688704 00000100:00100000:0.0:1789760725.399579:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493688704 00000100:00100000:0.0:1789760725.399709:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735626612635 time 735680470248 state 0x0 ref 1 00000100:00100000:0.0:1789760725.399725:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735680730664 time 735680470248 state 0x1 ref 1 00000100:00100000:0.0:1789760725.399758:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735680730664 time 735740793519 state 0x0 ref 1 00000100:00100000:0.0:1789760725.399843:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493688704:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760725.400013:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22642688, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:0.0:1789760725.400022:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04d52 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.400043:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760725.400048:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760725.400055:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04d52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.400069:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04d52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760725.449609:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972847 is committed 00002000:00010000:0.0:1789760725.449992:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22642688 -> 22646784 00010000:00010000:0.0:1789760725.450012:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04d52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.450026:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04d52 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.450034:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04d52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.450042:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760725.450047:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04d52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.450056:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04d52 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.450091:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04d52 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.450101:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04d52 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760725.450225:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493688704:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50449us (50809us total) trans 4294972847 rc 0/0 00000100:00100000:0.0:1789760725.450237:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5558 00000100:00100000:0.0:1789760725.450249:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735680730664 time 735740793519 state 0x2 ref 0 00000100:00100000:0.0:1789760725.461713:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760725.461857:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493688832 00000100:00100000:3.0:1789760725.461880:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493688832 00000100:00100000:3.0:1789760725.461957:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735680730664 time 735740793519 state 0x0 ref 1 00000100:00100000:3.0:1789760725.462026:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735741053935 time 735740793519 state 0x1 ref 1 00000100:00100000:3.0:1789760725.462059:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735741053935 time 735803093923 state 0x0 ref 1 00000100:00100000:3.0:1789760725.462074:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493688832:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760725.462150:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22646784, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:3.0:1789760725.462162:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04d59 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.462182:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760725.462188:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.462196:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04d59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.462212:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04d59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760725.487491:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972848 is committed 00002000:00010000:1.0:1789760725.487673:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22646784 -> 22650880 00010000:00010000:1.0:1789760725.487691:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04d59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.487705:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04d59 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.487716:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04d59 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.487730:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760725.487735:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04d59 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.487747:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04d59 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.487771:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04d59 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.487779:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04d59 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.488529:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493688832:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26447us (26819us total) trans 4294972848 rc 0/0 00000100:00100000:1.0:1789760725.488544:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5559 00000100:00100000:1.0:1789760725.488556:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735741053935 time 735803093923 state 0x2 ref 0 00000100:00100000:1.0:1789760725.497730:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760725.497941:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493688960 00000100:00100000:3.0:1789760725.497953:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493688960 00000100:00100000:3.0:1789760725.498057:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735741053935 time 735803093923 state 0x0 ref 1 00000100:00100000:3.0:1789760725.498076:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735803354339 time 735803093923 state 0x1 ref 1 00000100:00100000:3.0:1789760725.498113:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735803354339 time 735839147864 state 0x0 ref 1 00000100:00100000:3.0:1789760725.498127:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493688960:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760725.499023:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22650880, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:3.0:1789760725.499038:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04d60 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.499133:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760725.499140:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.499148:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04d60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.499166:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04d60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760725.542014:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972849 is committed 00002000:00010000:3.0:1789760725.542177:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22650880 -> 22654976 00010000:00010000:3.0:1789760725.542198:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04d60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.542219:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04d60 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.542230:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04d60 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.542239:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760725.542246:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04d60 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.542257:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04d60 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.542283:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04d60 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.542303:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04d60 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.542418:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493688960:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44286us (44697us total) trans 4294972849 rc 0/0 00000100:00100000:3.0:1789760725.542430:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5560 00000100:00100000:3.0:1789760725.542443:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735803354339 time 735839147864 state 0x2 ref 0 00000100:00100000:3.0:1789760725.549956:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760725.555759:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493689088 00000100:00100000:2.0:1789760725.555791:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493689088 00000100:00100000:3.0:1789760725.556104:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735803354339 time 735839147864 state 0x0 ref 1 00000100:00100000:3.0:1789760725.556126:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735839408280 time 735839147864 state 0x1 ref 1 00000100:00100000:3.0:1789760725.556170:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735839408280 time 735897204431 state 0x0 ref 1 00000100:00100000:3.0:1789760725.556187:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493689088:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760725.556253:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22654976, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:3.0:1789760725.556264:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04d67 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.556285:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760725.556290:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.556299:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04d67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.556315:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04d67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.562786:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760725.586024:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972850 is committed 00002000:00010000:1.0:1789760725.586249:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22654976 -> 22659072 00010000:00010000:1.0:1789760725.586267:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04d67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.586282:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04d67 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.586297:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04d67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.586306:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760725.586312:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04d67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.586321:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04d67 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.586344:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04d67 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.586352:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04d67 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.586473:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493689088:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30283us (36526us total) trans 4294972850 rc 0/0 00000100:00100000:1.0:1789760725.586486:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5561 00000100:00100000:1.0:1789760725.586498:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735839408280 time 735897204431 state 0x2 ref 0 00000100:00100000:3.0:1789760725.589105:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760725.589510:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493689216 00000100:00100000:1.0:1789760725.589523:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493689216 00000100:00100000:1.0:1789760725.589610:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735839408280 time 735897204431 state 0x0 ref 1 00000100:00100000:1.0:1789760725.589632:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735897464847 time 735897204431 state 0x1 ref 1 00000100:00100000:1.0:1789760725.589673:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735897464847 time 735930708458 state 0x0 ref 1 00000100:00100000:1.0:1789760725.589688:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493689216:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760725.589757:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22659072, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0xb007 00010000:00010000:1.0:1789760725.589768:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d6e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.589786:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760725.589792:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760725.589800:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.589818:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760725.623165:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972851 is committed 00002000:00010000:1.0:1789760725.623288:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22659072 -> 22663168 00002000:00010000:1.0:1789760725.623295:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760724 -> 1789760725 00002000:00010000:1.0:1789760725.623298:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760724 -> 1789760725 00010000:00010000:1.0:1789760725.623317:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.623331:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d6e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.623341:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.623351:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760725.623357:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.623368:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d6e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.623393:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d6e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.623403:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d6e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.623566:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493689216:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33872us (34479us total) trans 4294972851 rc 0/0 00000100:00100000:1.0:1789760725.623578:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5562 00000100:00100000:1.0:1789760725.623591:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735897464847 time 735930708458 state 0x2 ref 0 00000100:00100000:3.0:1789760725.626761:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760725.626849:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493689344 00000100:00100000:2.0:1789760725.626862:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493689344 00000100:00100000:2.0:1789760725.626938:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735897464847 time 735930708458 state 0x0 ref 1 00000100:00100000:2.0:1789760725.626957:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735930968874 time 735930708458 state 0x1 ref 1 00000100:00100000:2.0:1789760725.627008:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735930968874 time 735968044081 state 0x0 ref 1 00000100:00100000:2.0:1789760725.627023:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8555180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493689344:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760725.627096:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22663168, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:2.0:1789760725.627108:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04d75 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.627127:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760725.627133:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.627141:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04d75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.627158:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04d75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760725.668047:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972852 is committed 00002000:00010000:1.0:1789760725.668358:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22663168 -> 22667264 00010000:00010000:1.0:1789760725.668378:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04d75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.668394:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04d75 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.668412:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04d75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.668421:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760725.668428:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04d75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.668439:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04d75 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.668469:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04d75 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.668478:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04d75 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.668614:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8555180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493689344:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41574us (41848us total) trans 4294972852 rc 0/0 00000100:00100000:1.0:1789760725.668626:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5563 00000100:00100000:1.0:1789760725.668638:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735930968874 time 735968044081 state 0x2 ref 0 00000100:00100000:0.0:1789760725.674290:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760725.674469:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493689472 00000100:00100000:3.0:1789760725.674482:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493689472 00000100:00100000:3.0:1789760725.674561:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735930968874 time 735968044081 state 0x0 ref 1 00000100:00100000:3.0:1789760725.674580:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735968304497 time 735968044081 state 0x1 ref 1 00000100:00100000:3.0:1789760725.674754:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735968304497 time 736015788195 state 0x0 ref 1 00000100:00100000:3.0:1789760725.674780:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493689472:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760725.674860:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22667264, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:3.0:1789760725.674872:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04d7c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.674892:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760725.674898:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.674905:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04d7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.674923:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04d7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760725.714932:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22667264 -> 22671360 00000001:00080000:1.0:1789760725.715001:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972853 is committed 00010000:00010000:0.0:1789760725.715081:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04d7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.715098:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04d7c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.715108:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04d7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.715116:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760725.715122:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04d7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.715132:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04d7c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.715160:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04d7c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.715168:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04d7c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760725.715313:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493689472:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40539us (41031us total) trans 4294972853 rc 0/0 00000100:00100000:0.0:1789760725.715326:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5564 00000100:00100000:0.0:1789760725.715339:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735968304497 time 736015788195 state 0x2 ref 0 00000100:00100000:0.0:1789760725.720851:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760725.721564:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493689600 00000100:00100000:1.0:1789760725.721581:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493689600 00000100:00100000:1.0:1789760725.722524:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 735968304497 time 736015788195 state 0x0 ref 1 00000100:00100000:1.0:1789760725.722552:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736016048611 time 736015788195 state 0x1 ref 1 00000100:00100000:1.0:1789760725.722614:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736016048611 time 736063644931 state 0x0 ref 1 00000100:00100000:1.0:1789760725.722635:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493689600:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760725.723236:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22671360, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:1.0:1789760725.723250:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d83 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.723271:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760725.723276:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760725.723283:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.723301:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760725.786000:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972854 is committed 00002000:00010000:1.0:1789760725.786280:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22671360 -> 22675456 00010000:00010000:1.0:1789760725.786301:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.786316:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d83 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.786326:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.786339:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760725.786345:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.786355:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d83 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.786381:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d83 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.786390:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d83 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.786510:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493689600:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63874us (66199us total) trans 4294972854 rc 0/0 00000100:00100000:1.0:1789760725.786524:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5565 00000100:00100000:1.0:1789760725.786535:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736016048611 time 736063644931 state 0x2 ref 0 00000100:00100000:3.0:1789760725.789405:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760725.789572:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493689728 00000100:00100000:1.0:1789760725.789586:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493689728 00000100:00100000:1.0:1789760725.789683:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736016048611 time 736063644931 state 0x0 ref 1 00000100:00100000:1.0:1789760725.789702:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736063905347 time 736063644931 state 0x1 ref 1 00000100:00100000:1.0:1789760725.789741:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736063905347 time 736130776497 state 0x0 ref 1 00000100:00100000:1.0:1789760725.789759:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493689728:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760725.789830:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22675456, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:1.0:1789760725.789841:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04d8a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.789860:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760725.789866:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760725.789873:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04d8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.789890:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04d8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760725.834638:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972855 is committed 00002000:00010000:2.0:1789760725.834834:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22675456 -> 22679552 00010000:00010000:2.0:1789760725.834855:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04d8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.834871:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04d8a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.834880:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04d8a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.834889:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760725.834894:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04d8a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.834905:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04d8a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.834932:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04d8a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.834940:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04d8a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.835429:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493689728:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45663us (46030us total) trans 4294972855 rc 0/0 00000100:00100000:2.0:1789760725.835442:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5566 00000100:00100000:2.0:1789760725.835455:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736063905347 time 736130776497 state 0x2 ref 0 00000100:00100000:2.0:1789760725.838931:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760725.839263:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493689856 00000100:00100000:1.0:1789760725.839279:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493689856 00000100:00100000:1.0:1789760725.839373:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736063905347 time 736130776497 state 0x0 ref 1 00000100:00100000:1.0:1789760725.839396:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736131036913 time 736130776497 state 0x1 ref 1 00000100:00100000:1.0:1789760725.839433:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736131036913 time 736180468364 state 0x0 ref 1 00000100:00100000:1.0:1789760725.839447:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493689856:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760725.839526:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22679552, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:1.0:1789760725.839539:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d91 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.839566:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760725.839571:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760725.839578:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.839606:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760725.890332:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972856 is committed 00002000:00010000:1.0:1789760725.890458:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22679552 -> 22683648 00010000:00010000:1.0:1789760725.890480:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.890495:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d91 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.890505:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.890513:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760725.890518:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.890528:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d91 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.890552:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d91 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.890561:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04d91 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.890743:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493689856:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51290us (51825us total) trans 4294972856 rc 0/0 00000100:00100000:1.0:1789760725.890755:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5567 00000100:00100000:1.0:1789760725.890766:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736131036913 time 736180468364 state 0x2 ref 0 00000100:00100000:3.0:1789760725.893889:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760725.894003:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493689984 00000100:00100000:1.0:1789760725.894016:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493689984 00000100:00100000:1.0:1789760725.894100:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736131036913 time 736180468364 state 0x0 ref 1 00000100:00100000:1.0:1789760725.894119:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736180728780 time 736180468364 state 0x1 ref 1 00000100:00100000:1.0:1789760725.894145:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736180728780 time 736235180474 state 0x0 ref 1 00000100:00100000:1.0:1789760725.894160:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8556a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493689984:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760725.894224:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22683648, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:1.0:1789760725.894234:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04d98 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.894252:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760725.894258:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760725.894266:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04d98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.894291:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04d98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760725.935897:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972857 is committed 00002000:00010000:2.0:1789760725.936009:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22683648 -> 22687744 00010000:00010000:2.0:1789760725.936085:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04d98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.936099:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04d98 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.936108:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04d98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.936116:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760725.936121:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04d98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.936136:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04d98 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.936155:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04d98 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.936162:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04d98 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.936256:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8556a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493689984:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42093us (42379us total) trans 4294972857 rc 0/0 00000100:00100000:2.0:1789760725.936266:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5568 00000100:00100000:2.0:1789760725.936276:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736180728780 time 736235180474 state 0x2 ref 0 00000100:00100000:0.0:1789760725.943071:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760725.943187:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493690112 00000100:00100000:2.0:1789760725.943198:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493690112 00000100:00100000:2.0:1789760725.943251:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736180728780 time 736235180474 state 0x0 ref 1 00000100:00100000:2.0:1789760725.943266:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736235440890 time 736235180474 state 0x1 ref 1 00000100:00100000:2.0:1789760725.943299:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736235440890 time 736284334600 state 0x0 ref 1 00000100:00100000:2.0:1789760725.943312:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7aa7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493690112:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760725.943361:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22687744, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:2.0:1789760725.943370:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04d9f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.943387:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760725.943391:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.943398:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04d9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.943411:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04d9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760726.021215:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22687744 -> 22691840 00010000:00010000:1.0:1789760726.021235:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04d9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.021249:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04d9f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.021258:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04d9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.021266:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760726.021272:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04d9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.021282:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04d9f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.021304:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04d9f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.021311:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e04d9f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760726.021416:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7aa7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493690112:12345-192.168.203.60@tcp:4:dd.0 Request processed in 78103us (78351us total) trans 4294972858 rc 0/0 00000100:00100000:1.0:1789760726.021427:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5569 00000100:00100000:1.0:1789760726.021436:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736235440890 time 736284334600 state 0x2 ref 0 00000001:00080000:0.0:1789760726.021952:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972858 is committed 00000100:00100000:0.0:1789760726.029933:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760726.030112:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493690240 00000100:00100000:2.0:1789760726.030122:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493690240 00000100:00100000:2.0:1789760726.030178:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736235440890 time 736284334600 state 0x0 ref 1 00000100:00100000:2.0:1789760726.030194:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736284595016 time 736284334600 state 0x1 ref 1 00000100:00100000:2.0:1789760726.030225:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736284595016 time 736371260893 state 0x0 ref 1 00000100:00100000:2.0:1789760726.030238:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493690240:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760726.030288:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22691840, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:2.0:1789760726.030296:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04da6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.030313:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760726.030318:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760726.030324:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04da6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.030343:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04da6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760726.102650:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972859 is committed 00002000:00010000:2.0:1789760726.103064:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22691840 -> 22695936 00010000:00010000:2.0:1789760726.103079:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04da6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.103093:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04da6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.103102:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04da6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.103111:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760726.103115:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04da6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.103125:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04da6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.103144:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04da6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.103151:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04da6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760726.103246:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493690240:12345-192.168.203.60@tcp:4:dd.0 Request processed in 73004us (73318us total) trans 4294972859 rc 0/0 00000100:00100000:2.0:1789760726.103258:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5570 00000100:00100000:2.0:1789760726.103267:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736284595016 time 736371260893 state 0x2 ref 0 00000100:00100000:3.0:1789760726.107691:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760726.107793:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493690368 00000100:00100000:1.0:1789760726.107804:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493690368 00000100:00100000:1.0:1789760726.107858:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736284595016 time 736371260893 state 0x0 ref 1 00000100:00100000:1.0:1789760726.107872:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736371521309 time 736371260893 state 0x1 ref 1 00000100:00100000:1.0:1789760726.107909:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736371521309 time 736448943819 state 0x0 ref 1 00000100:00100000:1.0:1789760726.107922:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493690368:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760726.107999:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22695936, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:1.0:1789760726.108008:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04dad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.108024:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760726.108029:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760726.108035:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04dad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.108049:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04dad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760726.168176:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972860 is committed 00002000:00010000:0.0:1789760726.173643:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22695936 -> 22700032 00010000:00010000:0.0:1789760726.173670:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04dad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.173682:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04dad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.173691:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04dad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.173699:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760726.173704:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04dad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.173713:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04dad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.173732:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04dad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.173739:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04dad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.173845:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493690368:12345-192.168.203.60@tcp:4:dd.0 Request processed in 65921us (66160us total) trans 4294972860 rc 0/0 00000100:00100000:0.0:1789760726.173867:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5571 00000100:00100000:0.0:1789760726.173876:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736371521309 time 736448943819 state 0x2 ref 0 00000100:00100000:2.0:1789760726.178316:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760726.178399:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493690496 00000100:00100000:1.0:1789760726.178414:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493690496 00000100:00100000:1.0:1789760726.178473:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736371521309 time 736448943819 state 0x0 ref 1 00000100:00100000:1.0:1789760726.178488:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736449204235 time 736448943819 state 0x1 ref 1 00000100:00100000:1.0:1789760726.178521:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736449204235 time 736519557763 state 0x0 ref 1 00000100:00100000:1.0:1789760726.178533:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493690496:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760726.178580:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22700032, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:1.0:1789760726.178588:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04db4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.178613:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760726.178617:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760726.178624:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04db4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.178638:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04db4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760726.208761:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972861 is committed 00002000:00010000:0.0:1789760726.208872:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22700032 -> 22704128 00010000:00010000:0.0:1789760726.208888:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04db4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.208912:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04db4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.208927:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04db4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.208935:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760726.208940:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04db4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.208950:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04db4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.208997:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04db4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.209007:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04db4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.209088:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493690496:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30552us (30780us total) trans 4294972861 rc 0/0 00000100:00100000:0.0:1789760726.209099:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5572 00000100:00100000:0.0:1789760726.209109:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736449204235 time 736519557763 state 0x2 ref 0 00000100:00100000:2.0:1789760726.213875:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760726.214768:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493690624 00000100:00100000:3.0:1789760726.214779:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493690624 00000100:00100000:3.0:1789760726.214836:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736449204235 time 736519557763 state 0x0 ref 1 00000100:00100000:3.0:1789760726.214856:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736519818179 time 736519557763 state 0x1 ref 1 00000100:00100000:3.0:1789760726.214886:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736519818179 time 736555921328 state 0x0 ref 1 00000100:00100000:3.0:1789760726.214906:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493690624:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760726.214955:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22704128, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:3.0:1789760726.214991:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04dbb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.215008:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760726.215013:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.215020:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04dbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.215035:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04dbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760726.253919:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22704128 -> 22708224 00010000:00010000:1.0:1789760726.253941:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04dbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.253990:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04dbb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.254000:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04dbb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.254008:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760726.254013:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04dbb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.254023:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04dbb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.254043:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04dbb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.254051:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04dbb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760726.254135:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493690624:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39234us (40265us total) trans 4294972862 rc 0/0 00000100:00100000:1.0:1789760726.254145:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5573 00000100:00100000:1.0:1789760726.254155:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736519818179 time 736555921328 state 0x2 ref 0 00000001:00080000:0.0:1789760726.254941:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972862 is committed 00000100:00100000:3.0:1789760726.257193:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760726.257434:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493690752 00000100:00100000:2.0:1789760726.257445:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493690752 00000100:00100000:2.0:1789760726.257500:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736519818179 time 736555921328 state 0x0 ref 1 00000100:00100000:2.0:1789760726.257515:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736556181744 time 736555921328 state 0x1 ref 1 00000100:00100000:2.0:1789760726.257545:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736556181744 time 736598580281 state 0x0 ref 1 00000100:00100000:2.0:1789760726.257558:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493690752:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760726.257614:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22708224, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:2.0:1789760726.257625:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04dc2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.257641:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760726.257645:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760726.257713:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04dc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.257734:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04dc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760726.284980:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22708224 -> 22712320 00010000:00010000:3.0:1789760726.285001:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04dc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.285016:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04dc2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.285027:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04dc2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.285035:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760726.285041:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04dc2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.285051:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04dc2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.285071:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04dc2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.285079:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04dc2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.285180:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493690752:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27618us (27990us total) trans 4294972863 rc 0/0 00000100:00100000:3.0:1789760726.285192:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5574 00000100:00100000:3.0:1789760726.285203:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736556181744 time 736598580281 state 0x2 ref 0 00000100:00100000:3.0:1789760726.288476:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760726.288518:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493690880 00000100:00100000:3.0:1789760726.288528:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493690880 00000100:00100000:3.0:1789760726.288577:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736556181744 time 736598580281 state 0x0 ref 1 00000100:00100000:3.0:1789760726.288604:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736598840697 time 736598580281 state 0x1 ref 1 00000100:00100000:3.0:1789760726.288628:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736598840697 time 736629663684 state 0x0 ref 1 00000100:00100000:3.0:1789760726.288641:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493690880:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760726.288697:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22712320, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:3.0:1789760726.288706:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04dc9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.288722:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760726.288726:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.288732:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04dc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.288744:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04dc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760726.289764:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972863 is committed 00000001:00080000:1.0:1789760726.314490:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972864 is committed 00002000:00010000:3.0:1789760726.314762:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22712320 -> 22716416 00010000:00010000:3.0:1789760726.314778:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04dc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.314792:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04dc9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.314801:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04dc9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.314808:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760726.314813:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04dc9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.314821:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04dc9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.314841:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04dc9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.314849:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04dc9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.315000:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493690880:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26356us (26527us total) trans 4294972864 rc 0/0 00000100:00100000:3.0:1789760726.315012:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5575 00000100:00100000:3.0:1789760726.315022:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736598840697 time 736629663684 state 0x2 ref 0 00000100:00100000:1.0:1789760726.318370:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760726.318498:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493691008 00000100:00100000:0.0:1789760726.318508:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493691008 00000100:00100000:0.0:1789760726.318560:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736598840697 time 736629663684 state 0x0 ref 1 00000100:00100000:0.0:1789760726.318575:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736629924100 time 736629663684 state 0x1 ref 1 00000100:00100000:0.0:1789760726.318641:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736629924100 time 736659675626 state 0x0 ref 1 00000100:00100000:0.0:1789760726.318662:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493691008:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760726.318709:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22716416, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:0.0:1789760726.318718:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04dd0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.318731:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760726.318735:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.318741:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04dd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.318754:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04dd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760726.344173:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972865 is committed 00002000:00010000:3.0:1789760726.344325:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22716416 -> 22720512 00010000:00010000:3.0:1789760726.344342:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04dd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.344355:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04dd0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.344364:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04dd0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.344371:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760726.344377:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04dd0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.344387:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04dd0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.344405:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04dd0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.344413:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04dd0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.344500:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493691008:12345-192.168.203.60@tcp:4:dd.0 Request processed in 25836us (26133us total) trans 4294972865 rc 0/0 00000100:00100000:3.0:1789760726.344512:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5576 00000100:00100000:3.0:1789760726.344521:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736629924100 time 736659675626 state 0x2 ref 0 00000100:00100000:1.0:1789760726.348100:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760726.348238:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493691136 00000100:00100000:3.0:1789760726.348249:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493691136 00000100:00100000:3.0:1789760726.348307:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736629924100 time 736659675626 state 0x0 ref 1 00000100:00100000:3.0:1789760726.348322:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736659936042 time 736659675626 state 0x1 ref 1 00000100:00100000:3.0:1789760726.348572:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736659936042 time 736689395498 state 0x0 ref 1 00000100:00100000:3.0:1789760726.348589:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493691136:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760726.348742:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22720512, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:3.0:1789760726.348753:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04dd7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.348770:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760726.348774:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.348780:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04dd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.348793:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04dd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760726.386405:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972866 is committed 00002000:00010000:3.0:1789760726.386517:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22720512 -> 22724608 00010000:00010000:3.0:1789760726.386531:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04dd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.386543:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04dd7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.386551:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04dd7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.386558:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760726.386562:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04dd7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.386571:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04dd7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.386605:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04dd7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.386613:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04dd7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.386704:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493691136:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38114us (38608us total) trans 4294972866 rc 0/0 00000100:00100000:3.0:1789760726.386714:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5577 00000100:00100000:3.0:1789760726.386723:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736659936042 time 736689395498 state 0x2 ref 0 00000100:00100000:1.0:1789760726.390141:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760726.390218:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493691264 00000100:00100000:3.0:1789760726.390228:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493691264 00000100:00100000:3.0:1789760726.390278:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736659936042 time 736689395498 state 0x0 ref 1 00000100:00100000:3.0:1789760726.390293:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736689655914 time 736689395498 state 0x1 ref 1 00000100:00100000:3.0:1789760726.390328:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736689655914 time 736731363443 state 0x0 ref 1 00000100:00100000:3.0:1789760726.390339:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493691264:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760726.390386:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22724608, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:3.0:1789760726.390394:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dde lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.390409:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760726.390414:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.390419:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.390433:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760726.428362:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22724608 -> 22728704 00010000:00010000:3.0:1789760726.428383:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.428397:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dde lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.428406:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.428414:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760726.428419:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.428428:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dde lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.428448:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dde lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.428455:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dde lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.428549:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493691264:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38206us (38413us total) trans 4294972867 rc 0/0 00000100:00100000:3.0:1789760726.428560:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5578 00000100:00100000:3.0:1789760726.428568:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736689655914 time 736731363443 state 0x2 ref 0 00000001:00080000:2.0:1789760726.428660:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972867 is committed 00000100:00100000:1.0:1789760726.434723:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760726.434787:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493691392 00000100:00100000:0.0:1789760726.434797:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493691392 00000100:00100000:0.0:1789760726.434861:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736689655914 time 736731363443 state 0x0 ref 1 00000100:00100000:0.0:1789760726.434877:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736731623859 time 736731363443 state 0x1 ref 1 00000100:00100000:0.0:1789760726.434908:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736731623859 time 736775943652 state 0x0 ref 1 00000100:00100000:0.0:1789760726.434920:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493691392:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760726.434994:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22728704, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:0.0:1789760726.435003:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04de5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.435019:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760726.435023:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.435030:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04de5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.435043:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04de5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760726.467292:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972868 is committed 00002000:00010000:3.0:1789760726.467517:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22728704 -> 22732800 00010000:00010000:3.0:1789760726.467532:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04de5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.467546:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04de5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.467555:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04de5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.467561:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760726.467566:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04de5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.467574:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04de5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.467607:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04de5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.467620:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04de5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.467869:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493691392:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32794us (33000us total) trans 4294972868 rc 0/0 00000100:00100000:3.0:1789760726.467882:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5579 00000100:00100000:3.0:1789760726.467891:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736731623859 time 736775943652 state 0x2 ref 0 00000100:00100000:0.0:1789760726.472201:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760726.473032:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493691520 00000100:00100000:3.0:1789760726.473045:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493691520 00000100:00100000:3.0:1789760726.473102:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736731623859 time 736775943652 state 0x0 ref 1 00000100:00100000:3.0:1789760726.473117:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736776204068 time 736775943652 state 0x1 ref 1 00000100:00100000:3.0:1789760726.473211:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736776204068 time 736814245088 state 0x0 ref 1 00000100:00100000:3.0:1789760726.473224:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493691520:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760726.473273:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22732800, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:3.0:1789760726.473282:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.473298:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760726.473303:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.473309:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.473327:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760726.517816:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972869 is committed 00002000:00010000:3.0:1789760726.517949:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22732800 -> 22736896 00010000:00010000:3.0:1789760726.517992:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.518005:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.518013:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.518020:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760726.518025:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.518034:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.518069:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.518079:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.520262:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493691520:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47027us (48058us total) trans 4294972869 rc 0/0 00000100:00100000:3.0:1789760726.520278:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5580 00000100:00100000:3.0:1789760726.520288:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736776204068 time 736814245088 state 0x2 ref 0 00000100:00100000:1.0:1789760726.522806:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760726.522946:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493691648 00000100:00100000:3.0:1789760726.522956:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493691648 00000100:00100000:3.0:1789760726.523442:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736776204068 time 736814245088 state 0x0 ref 1 00000100:00100000:3.0:1789760726.523458:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736814505504 time 736814245088 state 0x1 ref 1 00000100:00100000:2.0:1789760726.523603:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736814505504 time 736864626101 state 0x0 ref 1 00000100:00100000:2.0:1789760726.523621:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493691648:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760726.523674:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22736896, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:2.0:1789760726.523682:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04df3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.523696:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760726.523702:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760726.523708:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04df3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.523721:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04df3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760726.588988:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22736896 -> 22740992 00010000:00010000:2.0:1789760726.589009:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04df3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.589022:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04df3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.589036:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04df3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.589044:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760726.589049:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04df3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.589059:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04df3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760726.589200:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972870 is committed 00010000:00010000:2.0:1789760726.589287:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04df3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.589297:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04df3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760726.589430:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493691648:12345-192.168.203.60@tcp:4:dd.0 Request processed in 65811us (66631us total) trans 4294972870 rc 0/0 00000100:00100000:2.0:1789760726.589441:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5581 00000100:00100000:2.0:1789760726.589450:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736814505504 time 736864626101 state 0x2 ref 0 00010000:00010000:3.0:1789760726.589626:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789760726.595327:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760726.595437:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493691776 00000100:00100000:3.0:1789760726.595447:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493691776 00000100:00100000:3.0:1789760726.595501:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736814505504 time 736864626101 state 0x0 ref 1 00000100:00100000:3.0:1789760726.595515:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736864886517 time 736864626101 state 0x1 ref 1 00000100:00100000:3.0:1789760726.595624:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736864886517 time 736936658674 state 0x0 ref 1 00000100:00100000:3.0:1789760726.595637:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493691776:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760726.595713:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22740992, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0xb007 00010000:00010000:3.0:1789760726.595722:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dfa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.595738:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760726.595742:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.595748:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.595772:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760726.639891:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22740992 -> 22745088 00002000:00010000:3.0:1789760726.639900:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760725 -> 1789760726 00002000:00010000:3.0:1789760726.639902:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760725 -> 1789760726 00010000:00010000:3.0:1789760726.639916:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.639929:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dfa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.639938:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dfa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.639945:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760726.639950:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dfa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.640027:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dfa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.640048:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dfa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.640056:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04dfa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.640146:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493691776:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44506us (44823us total) trans 4294972871 rc 0/0 00000100:00100000:3.0:1789760726.640156:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5582 00000100:00100000:3.0:1789760726.640166:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736864886517 time 736936658674 state 0x2 ref 0 00000001:00080000:0.0:1789760726.640314:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972871 is committed 00000100:00100000:0.0:1789760726.645524:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760726.645794:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493691904 00000100:00100000:3.0:1789760726.645806:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493691904 00000100:00100000:3.0:1789760726.645862:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736864886517 time 736936658674 state 0x0 ref 1 00000100:00100000:3.0:1789760726.645875:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736936919090 time 736936658674 state 0x1 ref 1 00000100:00100000:3.0:1789760726.646011:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736936919090 time 736987046142 state 0x0 ref 1 00000100:00100000:3.0:1789760726.646026:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493691904:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760726.646088:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22745088, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:3.0:1789760726.646097:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04e01 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.646113:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760726.646118:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.646125:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04e01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.646143:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04e01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760726.677272:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22745088 -> 22749184 00010000:00010000:3.0:1789760726.677292:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04e01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.677305:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04e01 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.677314:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04e01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.677322:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760726.677327:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04e01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.677337:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04e01 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.677359:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04e01 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.677367:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04e01 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.677463:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493691904:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31436us (31949us total) trans 4294972872 rc 0/0 00000100:00100000:3.0:1789760726.677474:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5583 00000100:00100000:3.0:1789760726.677484:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736936919090 time 736987046142 state 0x2 ref 0 00000001:00080000:0.0:1789760726.677585:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972872 is committed 00000100:00100000:2.0:1789760726.684047:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760726.684130:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493692032 00000100:00100000:0.0:1789760726.684141:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493692032 00000100:00100000:0.0:1789760726.684193:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736936919090 time 736987046142 state 0x0 ref 1 00000100:00100000:0.0:1789760726.684209:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736987306558 time 736987046142 state 0x1 ref 1 00000100:00100000:0.0:1789760726.684239:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736987306558 time 737025274413 state 0x0 ref 1 00000100:00100000:0.0:1789760726.684252:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493692032:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760726.684302:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22749184, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:0.0:1789760726.684310:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04e08 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.684325:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760726.684330:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.684337:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04e08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.684351:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04e08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760726.736213:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972873 is committed 00002000:00010000:3.0:1789760726.736322:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22749184 -> 22753280 00010000:00010000:3.0:1789760726.736337:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04e08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.736349:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04e08 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.736359:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04e08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.736366:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760726.736370:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04e08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.736379:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04e08 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.736398:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04e08 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.736410:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04e08 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.736507:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493692032:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52251us (52465us total) trans 4294972873 rc 0/0 00000100:00100000:3.0:1789760726.736517:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5584 00000100:00100000:3.0:1789760726.736527:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736987306558 time 737025274413 state 0x2 ref 0 00000100:00100000:2.0:1789760726.741913:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760726.742528:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493692160 00000100:00100000:0.0:1789760726.742539:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493692160 00000100:00100000:0.0:1789760726.742610:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 736987306558 time 737025274413 state 0x0 ref 1 00000100:00100000:0.0:1789760726.742626:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737025534829 time 737025274413 state 0x1 ref 1 00000100:00100000:0.0:1789760726.742662:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737025534829 time 737083697255 state 0x0 ref 1 00000100:00100000:0.0:1789760726.742806:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493692160:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760726.742858:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22753280, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:0.0:1789760726.742867:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04e0f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.742884:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760726.742888:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.742895:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04e0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.742909:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04e0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760726.800538:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22753280 -> 22757376 00010000:00010000:0.0:1789760726.800559:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04e0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.800573:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04e0f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.800583:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04e0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.800604:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760726.800609:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04e0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.800618:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04e0f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.800636:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04e0f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.800644:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04e0f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.801055:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493692160:12345-192.168.203.60@tcp:4:dd.0 Request processed in 58375us (59145us total) trans 4294972874 rc 0/0 00000100:00100000:0.0:1789760726.801066:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5585 00000100:00100000:0.0:1789760726.801076:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737025534829 time 737083697255 state 0x2 ref 0 00000001:00080000:1.0:1789760726.801425:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972874 is committed 00000100:00100000:1.0:1789760726.815176:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760726.815265:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493692288 00000100:00100000:3.0:1789760726.815275:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493692288 00000100:00100000:3.0:1789760726.815332:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737025534829 time 737083697255 state 0x0 ref 1 00000100:00100000:3.0:1789760726.815348:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737083957671 time 737083697255 state 0x1 ref 1 00000100:00100000:3.0:1789760726.815454:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737083957671 time 737156486629 state 0x0 ref 1 00000100:00100000:3.0:1789760726.815467:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493692288:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760726.815520:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22757376, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:3.0:1789760726.815529:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04e16 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.815546:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760726.815550:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.815557:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04e16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.815577:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04e16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760726.863134:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22757376 -> 22761472 00010000:00010000:0.0:1789760726.863154:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04e16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.863167:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04e16 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760726.863203:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972875 is committed 00010000:00010000:0.0:1789760726.863243:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04e16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.863251:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760726.863256:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04e16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.863266:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04e16 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.863286:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04e16 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.863300:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04e16 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.863482:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493692288:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48013us (48311us total) trans 4294972875 rc 0/0 00000100:00100000:0.0:1789760726.863491:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5586 00000100:00100000:0.0:1789760726.863498:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737083957671 time 737156486629 state 0x2 ref 0 00000100:00100000:2.0:1789760726.867670:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760726.867790:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493692416 00000100:00100000:3.0:1789760726.867801:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493692416 00000100:00100000:3.0:1789760726.867852:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737083957671 time 737156486629 state 0x0 ref 1 00000100:00100000:3.0:1789760726.867865:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737156747045 time 737156486629 state 0x1 ref 1 00000100:00100000:3.0:1789760726.867891:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737156747045 time 737208927452 state 0x0 ref 1 00000100:00100000:3.0:1789760726.867899:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493692416:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760726.867941:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22761472, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:3.0:1789760726.867947:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04e1d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.867990:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760726.867995:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.868001:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04e1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.868015:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04e1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760726.927897:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22761472 -> 22765568 00010000:00010000:2.0:1789760726.927933:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04e1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.927948:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04e1d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.927957:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04e1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.927986:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760726.927990:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04e1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.928001:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04e1d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.928023:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04e1d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.928031:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04e1d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760726.928132:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493692416:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60227us (60466us total) trans 4294972876 rc 0/0 00000100:00100000:2.0:1789760726.928144:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5587 00000100:00100000:2.0:1789760726.928154:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737156747045 time 737208927452 state 0x2 ref 0 00000001:00080000:1.0:1789760726.930305:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972876 is committed 00000100:00100000:2.0:1789760726.934383:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760726.937069:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493692544 00000100:00100000:0.0:1789760726.937087:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493692544 00000100:00100000:0.0:1789760726.937215:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737156747045 time 737208927452 state 0x0 ref 1 00000100:00100000:0.0:1789760726.937231:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737209187868 time 737208927452 state 0x1 ref 1 00000100:00100000:0.0:1789760726.938887:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737209187868 time 737279912928 state 0x0 ref 1 00000100:00100000:0.0:1789760726.938912:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493692544:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760726.938983:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22765568, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:0.0:1789760726.938991:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04e24 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.939006:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760726.939011:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.939017:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04e24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.939033:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04e24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.966838:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972877 is committed 00002000:00010000:3.0:1789760726.966943:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22765568 -> 22769664 00010000:00010000:3.0:1789760726.966987:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04e24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.967002:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04e24 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.967015:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04e24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.967023:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760726.967028:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04e24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.967037:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04e24 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.967059:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04e24 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.967066:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04e24 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.967241:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493692544:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28333us (32861us total) trans 4294972877 rc 0/0 00000100:00100000:3.0:1789760726.967251:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5588 00000100:00100000:3.0:1789760726.967260:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737209187868 time 737279912928 state 0x2 ref 0 00000100:00100000:3.0:1789760726.973238:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760726.973552:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493692672 00000100:00100000:0.0:1789760726.973562:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493692672 00000100:00100000:0.0:1789760726.973772:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737209187868 time 737279912928 state 0x0 ref 1 00000100:00100000:0.0:1789760726.973787:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737280173344 time 737279912928 state 0x1 ref 1 00000100:00100000:0.0:1789760726.973820:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737280173344 time 737314855731 state 0x0 ref 1 00000100:00100000:0.0:1789760726.973833:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493692672:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760726.973883:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22769664, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:0.0:1789760726.973894:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04e2b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.973909:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760726.973914:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.973920:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04e2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.973934:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04e2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760727.018329:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972878 is committed 00002000:00010000:3.0:1789760727.018569:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22769664 -> 22773760 00010000:00010000:3.0:1789760727.018583:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04e2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.018605:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04e2b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.018615:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04e2b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.018622:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760727.018627:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04e2b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.018636:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04e2b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.018665:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04e2b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.018673:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04e2b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.018764:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493692672:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44928us (45531us total) trans 4294972878 rc 0/0 00000100:00100000:3.0:1789760727.018778:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5589 00000100:00100000:3.0:1789760727.018788:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737280173344 time 737314855731 state 0x2 ref 0 00000100:00100000:2.0:1789760727.023605:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760727.023798:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493692800 00000100:00100000:0.0:1789760727.023808:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493692800 00000100:00100000:0.0:1789760727.023919:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737280173344 time 737314855731 state 0x0 ref 1 00000100:00100000:0.0:1789760727.023935:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737315116147 time 737314855731 state 0x1 ref 1 00000100:00100000:0.0:1789760727.023994:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737315116147 time 737365029387 state 0x0 ref 1 00000100:00100000:0.0:1789760727.024008:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493692800:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760727.024055:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22773760, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:0.0:1789760727.024064:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04e32 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.024079:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760727.024083:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.024089:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04e32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.024103:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04e32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760727.027274:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760727.061499:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972879 is committed 00002000:00010000:1.0:1789760727.061717:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22773760 -> 22777856 00010000:00010000:1.0:1789760727.061732:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04e32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.061745:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04e32 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.061754:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04e32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.061761:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760727.061766:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04e32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.061775:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04e32 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.061800:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04e32 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.061808:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e04e32 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.061930:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493692800:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37920us (38341us total) trans 4294972879 rc 0/0 00000100:00100000:1.0:1789760727.061940:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5590 00000100:00100000:1.0:1789760727.061950:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737315116147 time 737365029387 state 0x2 ref 0 00000100:00100000:2.0:1789760727.066293:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760727.066478:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493692928 00000100:00100000:3.0:1789760727.066488:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493692928 00000100:00100000:3.0:1789760727.066551:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737315116147 time 737365029387 state 0x0 ref 1 00000100:00100000:3.0:1789760727.066567:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737365289803 time 737365029387 state 0x1 ref 1 00000100:00100000:3.0:1789760727.066605:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737365289803 time 737407640863 state 0x0 ref 1 00000100:00100000:3.0:1789760727.066617:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493692928:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760727.066772:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22777856, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:3.0:1789760727.066781:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e39 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.066800:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760727.066805:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.066811:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.066825:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760727.103295:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972880 is committed 00002000:00010000:3.0:1789760727.103396:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22777856 -> 22781952 00010000:00010000:3.0:1789760727.103411:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.103424:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e39 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.103434:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.103442:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760727.103446:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.103455:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e39 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.103475:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e39 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.103921:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e39 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.104292:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493692928:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37670us (38005us total) trans 4294972880 rc 0/0 00000100:00100000:3.0:1789760727.104308:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5591 00000100:00100000:3.0:1789760727.104318:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737365289803 time 737407640863 state 0x2 ref 0 00000100:00100000:3.0:1789760727.112514:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760727.112626:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493693056 00000100:00100000:1.0:1789760727.112637:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493693056 00000100:00100000:1.0:1789760727.112697:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737365289803 time 737407640863 state 0x0 ref 1 00000100:00100000:1.0:1789760727.112713:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737407901279 time 737407640863 state 0x1 ref 1 00000100:00100000:1.0:1789760727.112749:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737407901279 time 737453784377 state 0x0 ref 1 00000100:00100000:1.0:1789760727.112762:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc17480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493693056:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760727.112821:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22781952, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:1.0:1789760727.112832:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04e40 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.112848:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760727.112853:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760727.112859:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04e40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.112874:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04e40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.143246:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760727.143293:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760727.143318:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760727.143344:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493693312 00000100:00100000:3.0:1789760727.143353:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493693312 00000100:00100000:3.0:1789760727.143405:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493693440 00000100:00100000:3.0:1789760727.143408:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493693440 00000100:00100000:3.0:1789760727.143430:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 467 00000100:00100000:3.0:1789760727.143440:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2369:x1876699493693312:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:3.0:1789760727.143512:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2369:x1876699493693312:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 73us (276us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760727.143520:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 467 00000100:00100000:3.0:1789760727.143536:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 468 00000100:00100000:3.0:1789760727.143542:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc14a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2370:x1876699493693440:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:3.0:1789760727.143565:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc14a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2370:x1876699493693440:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 24us (247us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760727.143572:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 468 00000100:00080000:3.0:1789760727.143598:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760727.144518:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493693184 00000100:00100000:2.0:1789760727.144527:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493693184 00000100:00100000:2.0:1789760727.144565:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 393 00000100:00100000:2.0:1789760727.144572:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d4d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2370:x1876699493693184:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:2.0:1789760727.144621:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d4d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2370:x1876699493693184:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 50us (1328us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760727.144628:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 393 00000001:00080000:3.0:1789760727.144738:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972881 is committed 00002000:00010000:1.0:1789760727.145372:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22781952 -> 22786048 00010000:00010000:1.0:1789760727.145387:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04e40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.145399:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04e40 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.145408:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04e40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.145414:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760727.145418:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04e40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.145426:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04e40 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.145448:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04e40 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.145455:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04e40 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.145512:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc17480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493693056:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32750us (33007us total) trans 4294972881 rc 0/0 00000100:00100000:1.0:1789760727.145521:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5592 00000100:00100000:1.0:1789760727.145530:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737407901279 time 737453784377 state 0x2 ref 0 00000100:00100000:3.0:1789760727.157074:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760727.157228:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493693568 00000100:00100000:0.0:1789760727.157239:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493693568 00000100:00100000:0.0:1789760727.157300:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737407901279 time 737453784377 state 0x0 ref 1 00000100:00100000:0.0:1789760727.157316:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737454044793 time 737453784377 state 0x1 ref 1 00000100:00100000:0.0:1789760727.157351:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737454044793 time 737498385888 state 0x0 ref 1 00000100:00100000:0.0:1789760727.157368:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493693568:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760727.157426:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22786048, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:0.0:1789760727.157437:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04e47 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.157458:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760727.157464:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.157471:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04e47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.157486:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04e47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760727.196932:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972882 is committed 00002000:00010000:3.0:1789760727.197139:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22786048 -> 22790144 00010000:00010000:3.0:1789760727.197154:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04e47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.197167:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04e47 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.197176:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04e47 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.197184:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760727.197188:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04e47 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.197197:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04e47 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.197216:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04e47 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.197224:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e04e47 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.197353:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493693568:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39984us (40294us total) trans 4294972882 rc 0/0 00000100:00100000:3.0:1789760727.197365:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5593 00000100:00100000:3.0:1789760727.197374:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737454044793 time 737498385888 state 0x2 ref 0 00000100:00100000:3.0:1789760727.200859:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760727.201005:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493693696 00000100:00100000:0.0:1789760727.201014:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493693696 00000100:00100000:0.0:1789760727.201067:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737454044793 time 737498385888 state 0x0 ref 1 00000100:00100000:0.0:1789760727.201081:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737498646304 time 737498385888 state 0x1 ref 1 00000100:00100000:0.0:1789760727.201115:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737498646304 time 737542150751 state 0x0 ref 1 00000100:00100000:0.0:1789760727.201129:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493693696:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760727.201178:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22790144, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:0.0:1789760727.201187:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04e4e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.201208:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760727.201214:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.201219:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04e4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.201232:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04e4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760727.233204:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972883 is committed 00002000:00010000:3.0:1789760727.233324:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22790144 -> 22794240 00010000:00010000:3.0:1789760727.233343:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04e4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.233357:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04e4e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.233367:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04e4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.233375:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760727.233380:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04e4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.233390:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04e4e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.233409:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04e4e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.233415:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e04e4e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.233507:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493693696:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32376us (32652us total) trans 4294972883 rc 0/0 00000100:00100000:3.0:1789760727.233518:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5594 00000100:00100000:3.0:1789760727.233528:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737498646304 time 737542150751 state 0x2 ref 0 00000100:00100000:2.0:1789760727.238574:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760727.238669:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493693824 00000100:00100000:3.0:1789760727.238679:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493693824 00000100:00100000:3.0:1789760727.238733:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737498646304 time 737542150751 state 0x0 ref 1 00000100:00100000:3.0:1789760727.238750:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737542411167 time 737542150751 state 0x1 ref 1 00000100:00100000:3.0:1789760727.238786:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737542411167 time 737579821744 state 0x0 ref 1 00000100:00100000:3.0:1789760727.238799:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493693824:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760727.238841:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22794240, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:3.0:1789760727.238848:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e55 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.238860:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760727.238864:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.238868:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.238880:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760727.293676:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22794240 -> 22798336 00010000:00010000:3.0:1789760727.293698:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.293711:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e55 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.293721:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.293729:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760727.293734:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.293742:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e55 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.293762:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e55 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.293770:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e55 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.293866:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493693824:12345-192.168.203.60@tcp:4:dd.0 Request processed in 55064us (55297us total) trans 4294972884 rc 0/0 00000100:00100000:3.0:1789760727.293878:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5595 00000100:00100000:3.0:1789760727.293889:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737542411167 time 737579821744 state 0x2 ref 0 00000001:00080000:1.0:1789760727.294496:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972884 is committed 00000100:00100000:2.0:1789760727.298616:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760727.298906:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493693952 00000100:00100000:1.0:1789760727.298914:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493693952 00000100:00100000:1.0:1789760727.299200:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737542411167 time 737579821744 state 0x0 ref 1 00000100:00100000:1.0:1789760727.299217:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737580082160 time 737579821744 state 0x1 ref 1 00000100:00100000:1.0:1789760727.299334:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737580082160 time 737640367957 state 0x0 ref 1 00000100:00100000:1.0:1789760727.299349:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493693952:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760727.299394:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22798336, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:1.0:1789760727.299403:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04e5c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.299419:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760727.299423:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760727.299430:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04e5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.299449:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04e5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760727.333352:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22798336 -> 22802432 00010000:00010000:2.0:1789760727.333372:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04e5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.333384:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04e5c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.333393:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04e5c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.333401:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760727.333406:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04e5c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.333415:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04e5c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.333435:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04e5c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.333443:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e04e5c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760727.333533:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493693952:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34182us (35013us total) trans 4294972885 rc 0/0 00000100:00100000:2.0:1789760727.333545:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5596 00000100:00100000:2.0:1789760727.333554:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737580082160 time 737640367957 state 0x2 ref 0 00000001:00080000:1.0:1789760727.334178:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972885 is committed 00000100:00100000:2.0:1789760727.339710:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760727.339830:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493694080 00000100:00100000:0.0:1789760727.339839:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493694080 00000100:00100000:0.0:1789760727.339888:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737580082160 time 737640367957 state 0x0 ref 1 00000100:00100000:0.0:1789760727.339902:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737640628373 time 737640367957 state 0x1 ref 1 00000100:00100000:0.0:1789760727.339930:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737640628373 time 737680965924 state 0x0 ref 1 00000100:00100000:0.0:1789760727.339941:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493694080:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760727.340017:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22802432, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:0.0:1789760727.340026:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04e63 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.340041:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760727.340045:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.340050:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04e63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.340068:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04e63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760727.392437:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972886 is committed 00002000:00010000:3.0:1789760727.392537:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22802432 -> 22806528 00010000:00010000:3.0:1789760727.392552:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04e63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.392570:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04e63 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.392579:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04e63 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.392594:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760727.392598:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04e63 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.392608:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04e63 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.392629:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04e63 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.392645:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e04e63 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.392740:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493694080:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52795us (53034us total) trans 4294972886 rc 0/0 00000100:00100000:3.0:1789760727.392750:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5597 00000100:00100000:3.0:1789760727.392760:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737640628373 time 737680965924 state 0x2 ref 0 00000100:00100000:2.0:1789760727.397872:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760727.398055:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493694208 00000100:00100000:3.0:1789760727.398068:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493694208 00000100:00100000:2.0:1789760727.399627:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737640628373 time 737680965924 state 0x0 ref 1 00000100:00100000:2.0:1789760727.399647:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737681226340 time 737680965924 state 0x1 ref 1 00000100:00100000:2.0:1789760727.399694:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737681226340 time 737740729852 state 0x0 ref 1 00000100:00100000:2.0:1789760727.399708:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493694208:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760727.399756:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22806528, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:2.0:1789760727.399765:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04e6a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.399781:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760727.399785:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760727.399790:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04e6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.399815:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04e6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760727.443853:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22806528 -> 22810624 00010000:00010000:2.0:1789760727.443876:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04e6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.443889:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04e6a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.443897:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04e6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.443905:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760727.443910:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04e6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.443919:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04e6a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.443939:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04e6a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.443947:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04e6a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760727.443951:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972887 is committed 00000100:00100000:2.0:1789760727.445567:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493694208:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45850us (47694us total) trans 4294972887 rc 0/0 00000100:00100000:2.0:1789760727.445581:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5598 00000100:00100000:2.0:1789760727.445599:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737681226340 time 737740729852 state 0x2 ref 0 00000100:00100000:2.0:1789760727.457742:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760727.458571:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493694336 00000100:00100000:3.0:1789760727.459203:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493694336 00000100:00100000:3.0:1789760727.459263:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737681226340 time 737740729852 state 0x0 ref 1 00000100:00100000:3.0:1789760727.459277:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737740990268 time 737740729852 state 0x1 ref 1 00000100:00100000:3.0:1789760727.459435:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737740990268 time 737800338656 state 0x0 ref 1 00000100:00100000:3.0:1789760727.459451:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493694336:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760727.459499:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22810624, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:3.0:1789760727.459508:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e71 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.459523:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760727.459528:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.459534:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.459548:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760727.513742:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972888 is committed 00002000:00010000:3.0:1789760727.513872:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22810624 -> 22814720 00010000:00010000:3.0:1789760727.513887:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.513901:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e71 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.513910:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e71 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.513917:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760727.513922:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e71 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.513931:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e71 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.513953:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e71 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.513999:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e71 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.514077:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493694336:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54626us (56341us total) trans 4294972888 rc 0/0 00000100:00100000:3.0:1789760727.514087:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5599 00000100:00100000:3.0:1789760727.514097:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737740990268 time 737800338656 state 0x2 ref 0 00000100:00100000:0.0:1789760727.518289:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760727.518462:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493694464 00000100:00100000:1.0:1789760727.518473:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493694464 00000100:00100000:1.0:1789760727.518529:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737740990268 time 737800338656 state 0x0 ref 1 00000100:00100000:1.0:1789760727.518545:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737800599072 time 737800338656 state 0x1 ref 1 00000100:00100000:1.0:1789760727.518644:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737800599072 time 737859670676 state 0x0 ref 1 00000100:00100000:1.0:1789760727.518659:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9621018a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493694464:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760727.518708:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22814720, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:1.0:1789760727.518717:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04e78 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.518733:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760727.518737:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760727.518743:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04e78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.518757:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04e78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760727.554685:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972889 is committed 00002000:00010000:1.0:1789760727.554842:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22814720 -> 22818816 00010000:00010000:1.0:1789760727.554860:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04e78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.554873:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04e78 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.554883:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04e78 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.554891:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760727.554896:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04e78 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.554904:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04e78 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.554923:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04e78 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.554931:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04e78 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.555146:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9621018a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493694464:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36405us (36783us total) trans 4294972889 rc 0/0 00000100:00100000:1.0:1789760727.555162:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5600 00000100:00100000:1.0:1789760727.555172:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737800599072 time 737859670676 state 0x2 ref 0 00000100:00100000:2.0:1789760727.560821:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760727.561186:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493694592 00000100:00100000:1.0:1789760727.561199:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493694592 00000100:00100000:1.0:1789760727.561264:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737800599072 time 737859670676 state 0x0 ref 1 00000100:00100000:1.0:1789760727.561282:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737859931092 time 737859670676 state 0x1 ref 1 00000100:00100000:1.0:1789760727.561351:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737859931092 time 737902385317 state 0x0 ref 1 00000100:00100000:1.0:1789760727.561365:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493694592:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760727.561424:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22818816, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:1.0:1789760727.561434:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04e7f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.561452:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760727.561457:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760727.561464:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04e7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.561479:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04e7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760727.589315:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972890 is committed 00002000:00010000:0.0:1789760727.589401:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22818816 -> 22822912 00010000:00010000:0.0:1789760727.589418:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04e7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.589433:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04e7f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.589442:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04e7f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.589450:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760727.589455:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04e7f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.589465:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04e7f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.589484:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04e7f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.589493:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04e7f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.589633:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493694592:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28265us (28817us total) trans 4294972890 rc 0/0 00000100:00100000:0.0:1789760727.589654:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5601 00000100:00100000:0.0:1789760727.589663:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737859931092 time 737902385317 state 0x2 ref 0 00000100:00100000:2.0:1789760727.593225:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760727.593529:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493694720 00000100:00100000:0.0:1789760727.593539:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493694720 00000100:00100000:0.0:1789760727.593606:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737859931092 time 737902385317 state 0x0 ref 1 00000100:00100000:0.0:1789760727.593622:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737902645733 time 737902385317 state 0x1 ref 1 00000100:00100000:0.0:1789760727.593659:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737902645733 time 737934694883 state 0x0 ref 1 00000100:00100000:0.0:1789760727.593672:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3661500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493694720:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760727.593721:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22822912, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0xb007 00010000:00010000:0.0:1789760727.593730:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04e86 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.593745:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760727.593750:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.593757:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04e86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.593771:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04e86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.610357:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760727.638040:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972891 is committed 00002000:00010000:0.0:1789760727.638201:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22822912 -> 22827008 00002000:00010000:0.0:1789760727.638206:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760726 -> 1789760727 00002000:00010000:0.0:1789760727.638208:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760726 -> 1789760727 00010000:00010000:0.0:1789760727.638221:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04e86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.638233:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04e86 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.638241:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04e86 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.638248:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760727.638253:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04e86 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.638263:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04e86 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.638281:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04e86 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.638290:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e04e86 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.638419:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3661500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493694720:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44744us (45199us total) trans 4294972891 rc 0/0 00000100:00100000:0.0:1789760727.638430:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5602 00000100:00100000:0.0:1789760727.638439:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737902645733 time 737934694883 state 0x2 ref 0 00000100:00100000:2.0:1789760727.642028:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760727.642187:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493694848 00000100:00100000:3.0:1789760727.642198:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493694848 00000100:00100000:3.0:1789760727.642260:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737902645733 time 737934694883 state 0x0 ref 1 00000100:00100000:3.0:1789760727.642278:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737934955299 time 737934694883 state 0x1 ref 1 00000100:00100000:3.0:1789760727.642311:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737934955299 time 737983345973 state 0x0 ref 1 00000100:00100000:3.0:1789760727.642324:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3662680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493694848:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760727.642374:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22827008, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:3.0:1789760727.642383:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e8d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.642398:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760727.642403:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.642410:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.642424:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760727.669579:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22827008 -> 22831104 00010000:00010000:2.0:1789760727.669607:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.669622:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e8d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.669630:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e8d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.669652:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760727.669658:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e8d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.669668:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e8d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.669689:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e8d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.669697:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04e8d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760727.669787:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3662680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493694848:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27459us (27763us total) trans 4294972892 rc 0/0 00000100:00100000:2.0:1789760727.669798:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5603 00000100:00100000:2.0:1789760727.669807:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737934955299 time 737983345973 state 0x2 ref 0 00000001:00080000:2.0:1789760727.669848:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972892 is committed 00000100:00100000:2.0:1789760727.674372:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760727.674515:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493694976 00000100:00100000:1.0:1789760727.674525:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493694976 00000100:00100000:1.0:1789760727.674591:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737934955299 time 737983345973 state 0x0 ref 1 00000100:00100000:1.0:1789760727.674608:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737983606389 time 737983345973 state 0x1 ref 1 00000100:00100000:1.0:1789760727.674630:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737983606389 time 738015666126 state 0x0 ref 1 00000100:00100000:1.0:1789760727.674652:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3662300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493694976:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760727.674704:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22831104, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:1.0:1789760727.674713:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04e94 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.674728:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760727.674733:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760727.674740:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04e94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.674754:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04e94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760727.701594:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972893 is committed 00002000:00010000:0.0:1789760727.703000:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22831104 -> 22835200 00010000:00010000:0.0:1789760727.703019:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04e94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.703034:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04e94 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.703043:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04e94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.703049:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760727.703054:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04e94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.703073:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04e94 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.703093:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04e94 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.703105:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04e94 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.703210:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3662300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493694976:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28556us (28843us total) trans 4294972893 rc 0/0 00000100:00100000:0.0:1789760727.703221:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5604 00000100:00100000:0.0:1789760727.703232:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737983606389 time 738015666126 state 0x2 ref 0 00000100:00100000:0.0:1789760727.706467:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760727.706610:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493695104 00000100:00100000:1.0:1789760727.706621:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493695104 00000100:00100000:1.0:1789760727.706692:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 737983606389 time 738015666126 state 0x0 ref 1 00000100:00100000:1.0:1789760727.706707:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738015926542 time 738015666126 state 0x1 ref 1 00000100:00100000:1.0:1789760727.706741:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738015926542 time 738047776290 state 0x0 ref 1 00000100:00100000:1.0:1789760727.706753:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9621018a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493695104:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760727.706809:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22835200, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:1.0:1789760727.706817:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04e9b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.706833:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760727.706838:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760727.706844:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04e9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.706858:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04e9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760727.746732:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22835200 -> 22839296 00010000:00010000:1.0:1789760727.746753:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04e9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.746765:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04e9b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.746774:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04e9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.746781:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760727.746786:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04e9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.746795:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04e9b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.746816:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04e9b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.746823:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04e9b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.746952:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9621018a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493695104:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40195us (40491us total) trans 4294972894 rc 0/0 00000100:00100000:1.0:1789760727.746993:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5605 00000100:00100000:1.0:1789760727.747003:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738015926542 time 738047776290 state 0x2 ref 0 00000001:00080000:2.0:1789760727.747376:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972894 is committed 00000100:00100000:2.0:1789760727.754525:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760727.754611:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493695232 00000100:00100000:0.0:1789760727.754622:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493695232 00000100:00100000:0.0:1789760727.754685:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738015926542 time 738047776290 state 0x0 ref 1 00000100:00100000:0.0:1789760727.754700:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738048036706 time 738047776290 state 0x1 ref 1 00000100:00100000:0.0:1789760727.754789:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738048036706 time 738095822099 state 0x0 ref 1 00000100:00100000:0.0:1789760727.754804:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493695232:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760727.754860:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22839296, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:0.0:1789760727.754871:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04ea2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.754889:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760727.754894:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.754901:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04ea2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.754916:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04ea2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760727.797223:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972895 is committed 00002000:00010000:0.0:1789760727.797460:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22839296 -> 22843392 00010000:00010000:0.0:1789760727.797477:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04ea2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.797490:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04ea2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.797498:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04ea2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.797505:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760727.797510:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04ea2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.797520:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04ea2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.797538:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04ea2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.797547:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e04ea2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.797658:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493695232:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42852us (43140us total) trans 4294972895 rc 0/0 00000100:00100000:0.0:1789760727.797669:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5606 00000100:00100000:0.0:1789760727.797679:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738048036706 time 738095822099 state 0x2 ref 0 00000100:00100000:0.0:1789760727.802187:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760727.802275:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493695360 00000100:00100000:2.0:1789760727.802285:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493695360 00000100:00100000:2.0:1789760727.802345:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738048036706 time 738095822099 state 0x0 ref 1 00000100:00100000:2.0:1789760727.802361:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738096082515 time 738095822099 state 0x1 ref 1 00000100:00100000:2.0:1789760727.802391:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738096082515 time 738143426855 state 0x0 ref 1 00000100:00100000:2.0:1789760727.802404:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493695360:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760727.802454:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22843392, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:2.0:1789760727.802464:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04ea9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.802480:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760727.802485:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760727.802491:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04ea9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.802505:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04ea9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760727.848788:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22843392 -> 22847488 00010000:00010000:3.0:1789760727.848818:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04ea9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.848831:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04ea9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.848840:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04ea9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.848848:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760727.848859:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04ea9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.848870:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04ea9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.848890:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04ea9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.848897:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e04ea9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.849030:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493695360:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46623us (46846us total) trans 4294972896 rc 0/0 00000100:00100000:3.0:1789760727.849039:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5607 00000100:00100000:3.0:1789760727.849049:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738096082515 time 738143426855 state 0x2 ref 0 00000001:00080000:0.0:1789760727.849128:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972896 is committed 00000100:00100000:2.0:1789760727.854557:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760727.854684:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493695488 00000100:00100000:1.0:1789760727.854694:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493695488 00000100:00100000:1.0:1789760727.854751:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738096082515 time 738143426855 state 0x0 ref 1 00000100:00100000:1.0:1789760727.854765:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738143687271 time 738143426855 state 0x1 ref 1 00000100:00100000:1.0:1789760727.854794:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738143687271 time 738195830746 state 0x0 ref 1 00000100:00100000:1.0:1789760727.854804:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493695488:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760727.854855:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22847488, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:1.0:1789760727.854864:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04eb0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.854880:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760727.854885:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760727.854891:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04eb0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.854905:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04eb0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760727.903089:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22847488 -> 22851584 00010000:00010000:1.0:1789760727.903110:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04eb0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760727.903120:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972897 is committed 00010000:00010000:1.0:1789760727.903124:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04eb0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.903147:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04eb0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.903154:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760727.903160:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04eb0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.903170:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04eb0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.903189:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04eb0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.903198:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04eb0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.903288:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493695488:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48480us (48736us total) trans 4294972897 rc 0/0 00000100:00100000:1.0:1789760727.903298:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5608 00000100:00100000:1.0:1789760727.903308:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738143687271 time 738195830746 state 0x2 ref 0 00000100:00100000:1.0:1789760727.922851:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760727.924029:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493695616 00000100:00100000:2.0:1789760727.924042:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493695616 00000100:00100000:2.0:1789760727.924105:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738143687271 time 738195830746 state 0x0 ref 1 00000100:00100000:2.0:1789760727.924121:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738196091162 time 738195830746 state 0x1 ref 1 00000100:00100000:2.0:1789760727.924155:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738196091162 time 738265188372 state 0x0 ref 1 00000100:00100000:2.0:1789760727.924168:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493695616:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760727.924211:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22851584, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:2.0:1789760727.924219:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04eb7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.924234:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760727.924238:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760727.924244:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04eb7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.924257:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04eb7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.930454:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4360700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519250048:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760727.930538:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760727.930838:0:6691:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519250048 00000100:00100000:0.0:1789760727.930848:0:6691:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519250048 00000100:00100000:2.0:1789760727.932224:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 194 00000100:00100000:2.0:1789760727.932251:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519250048:12345-0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1789760727.932699:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00000024:2.0:1789760727.932776:0:6690:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528285696 pending 1703936 free 3835691008 avail 3807379456 00000100:00100000:2.0:1789760727.932916:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519250048:12345-0@lo:13:osp-pre-0-0.0 Request processed in 679us (2377us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760727.932925:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 194 00000100:00100000:0.0:1789760727.933129:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4360700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519250048:0@lo:13:osp-pre-0-0.0 00000001:00080000:0.0:1789760727.948838:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972898 is committed 00002000:00010000:0.0:1789760727.948889:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22851584 -> 22855680 00010000:00010000:0.0:1789760727.948901:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04eb7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.948911:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04eb7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.948921:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04eb7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.948928:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760727.948934:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04eb7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.948945:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04eb7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.948995:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04eb7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.949005:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e04eb7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.949370:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493695616:12345-192.168.203.60@tcp:4:dd.0 Request processed in 25126us (26453us total) trans 4294972898 rc 0/0 00000100:00100000:0.0:1789760727.949381:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5609 00000100:00100000:0.0:1789760727.949391:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738196091162 time 738265188372 state 0x2 ref 0 00000100:00100000:0.0:1789760727.958612:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760727.958832:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493695744 00000100:00100000:2.0:1789760727.958844:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493695744 00000100:00100000:2.0:1789760727.958908:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738196091162 time 738265188372 state 0x0 ref 1 00000100:00100000:2.0:1789760727.958923:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738265448788 time 738265188372 state 0x1 ref 1 00000100:00100000:2.0:1789760727.958957:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738265448788 time 738299992217 state 0x0 ref 1 00000100:00100000:2.0:1789760727.959006:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493695744:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760727.959235:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22855680, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:2.0:1789760727.959243:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04ebe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.959256:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760727.959259:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760727.959264:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04ebe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.959273:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04ebe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760728.003383:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972899 is committed 00002000:00010000:1.0:1789760728.003489:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22855680 -> 22859776 00010000:00010000:1.0:1789760728.003505:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04ebe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.003519:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04ebe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.003527:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04ebe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.003534:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760728.003538:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04ebe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.003556:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04ebe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.003591:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04ebe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.003599:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e04ebe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.004198:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493695744:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45190us (45597us total) trans 4294972899 rc 0/0 00000100:00100000:1.0:1789760728.004210:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5610 00000100:00100000:1.0:1789760728.004220:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738265448788 time 738299992217 state 0x2 ref 0 00000100:00100000:2.0:1789760728.007775:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760728.009733:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493695872 00000100:00100000:3.0:1789760728.009748:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493695872 00000100:00100000:3.0:1789760728.009808:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738265448788 time 738299992217 state 0x0 ref 1 00000100:00100000:3.0:1789760728.009823:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738300252633 time 738299992217 state 0x1 ref 1 00000100:00100000:3.0:1789760728.009848:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738300252633 time 738350883726 state 0x0 ref 1 00000100:00100000:3.0:1789760728.009866:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493695872:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760728.009909:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22859776, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:3.0:1789760728.009917:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04ec5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.009932:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760728.009936:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760728.009942:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04ec5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.009955:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04ec5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760728.041001:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972900 is committed 00002000:00010000:1.0:1789760728.041346:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22859776 -> 22863872 00010000:00010000:1.0:1789760728.041362:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04ec5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.041374:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04ec5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.041383:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04ec5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.041391:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760728.041396:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04ec5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.041409:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04ec5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.041428:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04ec5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.041435:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04ec5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.041642:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493695872:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31765us (33863us total) trans 4294972900 rc 0/0 00000100:00100000:1.0:1789760728.041653:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5611 00000100:00100000:1.0:1789760728.041664:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738300252633 time 738350883726 state 0x2 ref 0 00000100:00100000:0.0:1789760728.046468:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760728.046521:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493696000 00000100:00100000:1.0:1789760728.046542:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493696000 00000100:00100000:1.0:1789760728.046615:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738300252633 time 738350883726 state 0x0 ref 1 00000100:00100000:1.0:1789760728.046630:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738351144142 time 738350883726 state 0x1 ref 1 00000100:00100000:1.0:1789760728.046664:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738351144142 time 738387699429 state 0x0 ref 1 00000100:00100000:1.0:1789760728.046676:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb04c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493696000:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760728.046723:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22863872, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:1.0:1789760728.046732:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04ecc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.046749:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760728.046754:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.046760:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04ecc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.046774:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04ecc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760728.075427:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22863872 -> 22867968 00010000:00010000:1.0:1789760728.075447:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04ecc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.075461:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04ecc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.075472:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04ecc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.075480:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760728.075485:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04ecc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.075494:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04ecc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.075514:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04ecc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.075522:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e04ecc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.075627:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb04c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493696000:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28947us (29162us total) trans 4294972901 rc 0/0 00000100:00100000:1.0:1789760728.075646:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5612 00000100:00100000:1.0:1789760728.075655:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738351144142 time 738387699429 state 0x2 ref 0 00000001:00080000:2.0:1789760728.077055:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972901 is committed 00000100:00100000:0.0:1789760728.080621:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760728.080790:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493696128 00000100:00100000:1.0:1789760728.080800:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493696128 00000100:00100000:1.0:1789760728.080859:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738351144142 time 738387699429 state 0x0 ref 1 00000100:00100000:1.0:1789760728.080874:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738387959845 time 738387699429 state 0x1 ref 1 00000100:00100000:1.0:1789760728.080904:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738387959845 time 738421939291 state 0x0 ref 1 00000100:00100000:1.0:1789760728.080917:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493696128:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760728.080994:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22867968, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:1.0:1789760728.081004:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04ed3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.081019:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760728.081024:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.081030:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04ed3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.081064:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04ed3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760728.110737:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22867968 -> 22872064 00010000:00010000:0.0:1789760728.110756:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04ed3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.110768:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04ed3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.110782:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04ed3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.110790:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760728.110795:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04ed3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.110804:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04ed3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.110822:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04ed3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.110830:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e04ed3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.110921:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493696128:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30002us (30307us total) trans 4294972902 rc 0/0 00000100:00100000:0.0:1789760728.110932:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5613 00000100:00100000:0.0:1789760728.110942:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738387959845 time 738421939291 state 0x2 ref 0 00000001:00080000:2.0:1789760728.111401:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972902 is committed 00000100:00100000:0.0:1789760728.120056:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760728.120370:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493696256 00000100:00100000:2.0:1789760728.120380:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493696256 00000100:00100000:2.0:1789760728.120435:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738387959845 time 738421939291 state 0x0 ref 1 00000100:00100000:2.0:1789760728.120451:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738422199707 time 738421939291 state 0x1 ref 1 00000100:00100000:2.0:1789760728.120483:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738422199707 time 738461518767 state 0x0 ref 1 00000100:00100000:2.0:1789760728.120509:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493696256:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760728.120566:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22872064, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:2.0:1789760728.120584:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04eda lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.120604:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760728.120609:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760728.120615:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04eda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.120630:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04eda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760728.141135:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972903 is committed 00002000:00010000:0.0:1789760728.141191:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22872064 -> 22876160 00010000:00010000:0.0:1789760728.141215:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04eda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.141237:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04eda lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.141248:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04eda lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.141257:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760728.141264:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04eda lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.141275:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04eda lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.141300:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04eda lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.141309:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e04eda lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.141523:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493696256:12345-192.168.203.60@tcp:4:dd.0 Request processed in 21026us (21473us total) trans 4294972903 rc 0/0 00000100:00100000:0.0:1789760728.141534:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5614 00000100:00100000:0.0:1789760728.141545:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738422199707 time 738461518767 state 0x2 ref 0 00000100:00100000:1.0:1789760728.147912:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760728.147995:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493696384 00000100:00100000:3.0:1789760728.148004:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493696384 00000100:00100000:3.0:1789760728.148233:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738422199707 time 738461518767 state 0x0 ref 1 00000100:00100000:3.0:1789760728.148252:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738461779183 time 738461518767 state 0x1 ref 1 00000100:00100000:3.0:1789760728.148283:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738461779183 time 738489318088 state 0x0 ref 1 00000100:00100000:3.0:1789760728.148295:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493696384:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760728.148342:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22876160, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:3.0:1789760728.148351:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04ee1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.148371:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760728.148376:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760728.148381:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04ee1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.148394:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04ee1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760728.175490:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972904 is committed 00002000:00010000:0.0:1789760728.175655:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22876160 -> 22880256 00010000:00010000:0.0:1789760728.175670:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04ee1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.175683:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04ee1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.175691:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04ee1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.175699:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760728.175704:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04ee1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.175714:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04ee1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.175733:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04ee1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.175738:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e04ee1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.175814:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493696384:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27512us (27903us total) trans 4294972904 rc 0/0 00000100:00100000:0.0:1789760728.175825:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5615 00000100:00100000:0.0:1789760728.175833:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738461779183 time 738489318088 state 0x2 ref 0 00000100:00100000:2.0:1789760728.180372:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760728.180589:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493696512 00000100:00100000:1.0:1789760728.180600:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493696512 00000100:00100000:1.0:1789760728.180664:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738461779183 time 738489318088 state 0x0 ref 1 00000100:00100000:1.0:1789760728.180678:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738489578504 time 738489318088 state 0x1 ref 1 00000100:00100000:1.0:1789760728.180709:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738489578504 time 738521745085 state 0x0 ref 1 00000100:00100000:1.0:1789760728.180722:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493696512:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760728.180772:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22880256, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:1.0:1789760728.180781:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04ee8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.180797:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760728.180801:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.180808:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04ee8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.180821:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04ee8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760728.214491:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972905 is committed 00002000:00010000:1.0:1789760728.215002:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22880256 -> 22884352 00010000:00010000:1.0:1789760728.215018:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04ee8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.215032:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04ee8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.215040:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04ee8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.215048:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760728.215053:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04ee8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.215062:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04ee8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.215083:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04ee8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.215090:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04ee8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.215238:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493696512:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34514us (34869us total) trans 4294972905 rc 0/0 00000100:00100000:1.0:1789760728.215252:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5616 00000100:00100000:1.0:1789760728.215272:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738489578504 time 738521745085 state 0x2 ref 0 00000100:00100000:2.0:1789760728.222505:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760728.222676:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493696640 00000100:00100000:0.0:1789760728.222686:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493696640 00000100:00100000:0.0:1789760728.222735:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738489578504 time 738521745085 state 0x0 ref 1 00000100:00100000:0.0:1789760728.222750:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738522005501 time 738521745085 state 0x1 ref 1 00000100:00100000:0.0:1789760728.222783:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738522005501 time 738563818717 state 0x0 ref 1 00000100:00100000:0.0:1789760728.222795:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493696640:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760728.222842:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22884352, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:0.0:1789760728.222850:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04eef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.222866:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760728.222871:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.222877:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04eef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.222890:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04eef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.262361:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972906 is committed 00002000:00010000:1.0:1789760728.262619:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22884352 -> 22888448 00010000:00010000:1.0:1789760728.262645:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04eef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.262657:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04eef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.262665:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04eef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.262672:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760728.262676:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04eef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.262685:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04eef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.262704:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04eef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.262711:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04eef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.262807:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493696640:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40009us (40306us total) trans 4294972906 rc 0/0 00000100:00100000:1.0:1789760728.262818:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5617 00000100:00100000:1.0:1789760728.262830:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738522005501 time 738563818717 state 0x2 ref 0 00000100:00100000:0.0:1789760728.266533:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760728.268202:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493696768 00000100:00100000:2.0:1789760728.268218:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493696768 00000100:00100000:2.0:1789760728.268284:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738522005501 time 738563818717 state 0x0 ref 1 00000100:00100000:2.0:1789760728.268368:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738564079133 time 738563818717 state 0x1 ref 1 00000100:00100000:2.0:1789760728.268407:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738564079133 time 738609442651 state 0x0 ref 1 00000100:00100000:2.0:1789760728.268421:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493696768:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760728.268470:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22888448, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:2.0:1789760728.268479:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04ef6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.268499:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760728.268503:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760728.268508:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04ef6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.268520:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04ef6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760728.334625:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972907 is committed 00002000:00010000:0.0:1789760728.334697:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22888448 -> 22892544 00010000:00010000:0.0:1789760728.334711:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04ef6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.334730:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04ef6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.334739:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04ef6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.334747:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760728.334752:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04ef6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.334760:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04ef6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.334778:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04ef6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.334786:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e04ef6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.334867:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493696768:12345-192.168.203.60@tcp:4:dd.0 Request processed in 66445us (68339us total) trans 4294972907 rc 0/0 00000100:00100000:0.0:1789760728.334878:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5618 00000100:00100000:0.0:1789760728.334887:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738564079133 time 738609442651 state 0x2 ref 0 00000100:00100000:2.0:1789760728.337933:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760728.338051:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493696896 00000100:00100000:3.0:1789760728.338061:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493696896 00000100:00100000:3.0:1789760728.338116:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738564079133 time 738609442651 state 0x0 ref 1 00000100:00100000:3.0:1789760728.338133:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738609703067 time 738609442651 state 0x1 ref 1 00000100:00100000:3.0:1789760728.338161:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738609703067 time 738679196823 state 0x0 ref 1 00000100:00100000:3.0:1789760728.338172:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493696896:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760728.338221:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22892544, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:3.0:1789760728.338231:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04efd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.338247:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760728.338251:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760728.338257:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04efd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.338272:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04efd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760728.377708:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22892544 -> 22896640 00010000:00010000:1.0:1789760728.377727:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04efd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.377741:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04efd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.377819:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04efd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.377828:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760728.377834:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04efd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.377843:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04efd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.377863:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04efd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.377876:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e04efd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.378012:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493696896:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39836us (40084us total) trans 4294972908 rc 0/0 00000100:00100000:1.0:1789760728.378022:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5619 00000100:00100000:1.0:1789760728.378032:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738609703067 time 738679196823 state 0x2 ref 0 00000001:00080000:2.0:1789760728.378712:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972908 is committed 00000100:00100000:2.0:1789760728.383499:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760728.383664:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493697024 00000100:00100000:1.0:1789760728.383674:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493697024 00000100:00100000:1.0:1789760728.383729:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738609703067 time 738679196823 state 0x0 ref 1 00000100:00100000:1.0:1789760728.383744:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738679457239 time 738679196823 state 0x1 ref 1 00000100:00100000:1.0:1789760728.383770:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738679457239 time 738724805987 state 0x0 ref 1 00000100:00100000:1.0:1789760728.383783:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493697024:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760728.383832:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22896640, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:1.0:1789760728.383840:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04f04 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.383856:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760728.383861:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.383867:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04f04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.383887:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04f04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760728.429791:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22896640 -> 22900736 00010000:00010000:0.0:1789760728.429812:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04f04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.429826:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04f04 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.429834:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04f04 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.429841:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760728.429846:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04f04 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.429855:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04f04 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.429873:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04f04 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.429881:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e04f04 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.430012:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493697024:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46226us (46518us total) trans 4294972909 rc 0/0 00000100:00100000:0.0:1789760728.430023:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5620 00000100:00100000:0.0:1789760728.430034:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738679457239 time 738724805987 state 0x2 ref 0 00000001:00080000:3.0:1789760728.430143:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972909 is committed 00000100:00100000:3.0:1789760728.435211:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760728.435358:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493697152 00000100:00100000:1.0:1789760728.435369:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493697152 00000100:00100000:1.0:1789760728.435419:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738679457239 time 738724805987 state 0x0 ref 1 00000100:00100000:1.0:1789760728.435434:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738725066403 time 738724805987 state 0x1 ref 1 00000100:00100000:1.0:1789760728.435464:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738725066403 time 738776499961 state 0x0 ref 1 00000100:00100000:1.0:1789760728.435477:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493697152:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760728.435527:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22900736, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:1.0:1789760728.435536:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f0b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.435552:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760728.435557:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.435563:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.435586:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760728.442136:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4d88700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3306:1876699519250176:0@lo:400:kworker.0 00000100:00100000:3.0:1789760728.442220:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760728.442331:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519250176 00000100:00100000:0.0:1789760728.442340:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519250176 00000100:00100000:0.0:1789760728.442392:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 394 00000100:00100000:0.0:1789760728.442402:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d4f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3306:x1876699519250176:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760728.442588:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620fbbed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3305:1876699519250560:0@lo:400:kworker.0 00000100:00100000:1.0:1789760728.442624:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760728.442785:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d4f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3306:x1876699519250176:12345-0@lo:400:kworker.0 Request processed in 383us (563us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760728.442793:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 394 00000100:00100000:0.0:1789760728.442827:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519250560 00000100:00100000:0.0:1789760728.442831:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519250560 00000100:00100000:0.0:1789760728.442872:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 469 00000100:00100000:0.0:1789760728.442879:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3305:x1876699519250560:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760728.442915:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3305:x1876699519250560:12345-0@lo:400:kworker.0 Request processed in 36us (289us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760728.442921:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 469 00000100:00100000:0.0:1789760728.442953:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620fbbed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3305:1876699519250560:0@lo:400:kworker.0 00000100:00080000:1.0:1789760728.443032:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760728.443113:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620fbbece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876699519250304:0@lo:400:kworker.0 00000100:00100000:2.0:1789760728.443143:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760728.443171:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760728.443176:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620fbbef100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3303:1876699519250432:0@lo:400:kworker.0 00000100:00100000:2.0:1789760728.443198:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760728.443226:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519250304 00000100:00100000:2.0:1789760728.443230:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519250304 00000100:00100000:2.0:1789760728.443254:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519250432 00000100:00100000:2.0:1789760728.443257:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519250432 00000100:00100000:2.0:1789760728.443281:0:9460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 470 00000100:00100000:2.0:1789760728.443286:0:9460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c89f8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876699519250304:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760728.443319:0:9460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c89f8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876699519250304:12345-0@lo:400:kworker.0 Request processed in 33us (175us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760728.443326:0:9460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 470 00000100:00100000:2.0:1789760728.443340:0:9460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 471 00000100:00100000:2.0:1789760728.443347:0:9460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c89f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876699519250432:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760728.443373:0:9460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c89f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876699519250432:12345-0@lo:400:kworker.0 Request processed in 27us (174us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760728.443378:0:9460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 471 00000100:00080000:2.0:1789760728.443396:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760728.443412:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620fbbef100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3303:1876699519250432:0@lo:400:kworker.0 00000100:00100000:2.0:1789760728.443421:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620fbbece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876699519250304:0@lo:400:kworker.0 00000100:00100000:0.0:1789760728.443514:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789760728.443523:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620fbbed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519250688:0@lo:400:kworker.0 00000100:00100000:0.0:1789760728.443551:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760728.443691:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519250688 00000100:00100000:2.0:1789760728.443695:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519250688 00000100:00100000:2.0:1789760728.443918:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 170 00000100:00100000:2.0:1789760728.443927:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519250688:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760728.443990:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519250688:12345-0@lo:400:kworker.0 Request processed in 66us (438us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760728.443998:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 170 00000100:00100000:3.0:1789760728.444068:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4d88700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3306:1876699519250176:0@lo:400:kworker.0 00000100:00080000:1.0:1789760728.444133:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789760728.444344:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620fbbed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519250688:0@lo:400:kworker.0 00002000:00010000:1.0:1789760728.466013:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22900736 -> 22904832 00010000:00010000:1.0:1789760728.466032:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.466046:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f0b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.466057:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.466065:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760728.466070:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.466079:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f0b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.466100:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f0b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.466107:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f0b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.466201:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493697152:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30721us (30994us total) trans 4294972910 rc 0/0 00000100:00100000:1.0:1789760728.466212:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5621 00000100:00100000:1.0:1789760728.466222:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738725066403 time 738776499961 state 0x2 ref 0 00000001:00080000:1.0:1789760728.466369:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972910 is committed 00000100:00100000:0.0:1789760728.470275:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760728.470349:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493697280 00000100:00100000:1.0:1789760728.470359:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493697280 00000100:00100000:1.0:1789760728.470411:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738725066403 time 738776499961 state 0x0 ref 1 00000100:00100000:1.0:1789760728.470427:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738776760377 time 738776499961 state 0x1 ref 1 00000100:00100000:1.0:1789760728.470455:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738776760377 time 738811491477 state 0x0 ref 1 00000100:00100000:1.0:1789760728.470467:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493697280:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760728.470518:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22904832, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:1.0:1789760728.470528:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04f12 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.470545:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760728.470550:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.470557:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04f12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.470573:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04f12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.502317:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972911 is committed 00002000:00010000:2.0:1789760728.502474:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22904832 -> 22908928 00010000:00010000:2.0:1789760728.502489:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04f12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.502502:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04f12 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.502510:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04f12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.502517:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760728.502522:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04f12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.502530:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04f12 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.502550:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04f12 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.502558:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04f12 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760728.502769:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493697280:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32299us (32499us total) trans 4294972911 rc 0/0 00000100:00100000:2.0:1789760728.502781:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5622 00000100:00100000:2.0:1789760728.502790:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738776760377 time 738811491477 state 0x2 ref 0 00000100:00100000:1.0:1789760728.506794:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760728.506874:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493697408 00000100:00100000:3.0:1789760728.506884:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493697408 00000100:00100000:3.0:1789760728.506940:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738776760377 time 738811491477 state 0x0 ref 1 00000100:00100000:3.0:1789760728.506956:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738811751893 time 738811491477 state 0x1 ref 1 00000100:00100000:3.0:1789760728.507208:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738811751893 time 738848242668 state 0x0 ref 1 00000100:00100000:3.0:1789760728.507222:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493697408:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760728.507276:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22908928, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:3.0:1789760728.507285:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04f19 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.507302:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760728.507307:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760728.507314:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04f19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.507328:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04f19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760728.539433:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972912 is committed 00002000:00010000:0.0:1789760728.539529:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22908928 -> 22913024 00010000:00010000:0.0:1789760728.539544:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04f19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.539557:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04f19 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.539583:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04f19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.539592:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760728.539598:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04f19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.539608:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04f19 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.539635:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04f19 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.539645:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e04f19 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.539736:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493697408:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32512us (32950us total) trans 4294972912 rc 0/0 00000100:00100000:0.0:1789760728.539747:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5623 00000100:00100000:0.0:1789760728.539757:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738811751893 time 738848242668 state 0x2 ref 0 00000100:00100000:3.0:1789760728.542697:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760728.543583:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493697536 00000100:00100000:0.0:1789760728.543594:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493697536 00000100:00100000:0.0:1789760728.543652:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738811751893 time 738848242668 state 0x0 ref 1 00000100:00100000:0.0:1789760728.543667:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738848503084 time 738848242668 state 0x1 ref 1 00000100:00100000:0.0:1789760728.543700:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738848503084 time 738884735714 state 0x0 ref 1 00000100:00100000:0.0:1789760728.543711:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493697536:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760728.543760:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22913024, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:0.0:1789760728.543768:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04f20 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.543785:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760728.543790:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.543796:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04f20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.543809:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04f20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.570459:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972913 is committed 00002000:00010000:2.0:1789760728.570677:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22913024 -> 22917120 00010000:00010000:2.0:1789760728.570691:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04f20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.570702:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04f20 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.570710:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04f20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.570722:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760728.570736:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04f20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.570746:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04f20 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.570765:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04f20 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.570772:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e04f20 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760728.570902:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493697536:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27187us (28207us total) trans 4294972913 rc 0/0 00000100:00100000:2.0:1789760728.570913:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5624 00000100:00100000:2.0:1789760728.570924:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738848503084 time 738884735714 state 0x2 ref 0 00000100:00100000:2.0:1789760728.574384:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760728.574491:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493697664 00000100:00100000:1.0:1789760728.574501:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493697664 00000100:00100000:1.0:1789760728.574558:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738848503084 time 738884735714 state 0x0 ref 1 00000100:00100000:1.0:1789760728.574585:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738884996130 time 738884735714 state 0x1 ref 1 00000100:00100000:1.0:1789760728.574668:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738884996130 time 738915702893 state 0x0 ref 1 00000100:00100000:1.0:1789760728.574681:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493697664:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760728.574730:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22917120, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0xb007 00010000:00010000:1.0:1789760728.574739:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f27 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.574755:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760728.574760:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.574766:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.574780:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760728.616181:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22917120 -> 22921216 00002000:00010000:1.0:1789760728.616191:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760727 -> 1789760728 00002000:00010000:1.0:1789760728.616194:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760727 -> 1789760728 00010000:00010000:1.0:1789760728.616208:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.616227:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f27 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.616236:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f27 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.616244:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760728.616249:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f27 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.616258:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f27 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.616269:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972914 is committed 00010000:00010000:1.0:1789760728.616278:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f27 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.616285:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f27 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.616386:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493697664:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41701us (42005us total) trans 4294972914 rc 0/0 00000100:00100000:1.0:1789760728.616398:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5625 00000100:00100000:1.0:1789760728.616408:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738884996130 time 738915702893 state 0x2 ref 0 00000100:00100000:0.0:1789760728.625335:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760728.625750:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493697792 00000100:00100000:1.0:1789760728.625761:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493697792 00000100:00100000:1.0:1789760728.625816:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738884996130 time 738915702893 state 0x0 ref 1 00000100:00100000:1.0:1789760728.625831:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738915963309 time 738915702893 state 0x1 ref 1 00000100:00100000:1.0:1789760728.625864:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738915963309 time 738966899520 state 0x0 ref 1 00000100:00100000:1.0:1789760728.625877:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493697792:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760728.625926:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22921216, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:1.0:1789760728.625934:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f2e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.625958:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760728.625977:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.625982:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.625998:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.634630:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760728.679215:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972915 is committed 00002000:00010000:1.0:1789760728.679862:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22921216 -> 22925312 00010000:00010000:1.0:1789760728.679876:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.679889:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f2e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.679903:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.679910:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760728.679915:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.679924:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f2e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.679947:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f2e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.679955:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f2e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.680080:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493697792:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54201us (54753us total) trans 4294972915 rc 0/0 00000100:00100000:1.0:1789760728.680090:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5626 00000100:00100000:1.0:1789760728.680099:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738915963309 time 738966899520 state 0x2 ref 0 00000100:00100000:0.0:1789760728.687168:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760728.687254:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493697920 00000100:00100000:3.0:1789760728.687262:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493697920 00000100:00100000:3.0:1789760728.687315:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738915963309 time 738966899520 state 0x0 ref 1 00000100:00100000:3.0:1789760728.687329:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738967159936 time 738966899520 state 0x1 ref 1 00000100:00100000:3.0:1789760728.687356:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738967159936 time 739028391726 state 0x0 ref 1 00000100:00100000:3.0:1789760728.687468:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493697920:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760728.687516:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22925312, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:3.0:1789760728.687541:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f35 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.687558:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760728.687562:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760728.687568:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.687597:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760728.735342:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972916 is committed 00002000:00010000:3.0:1789760728.735505:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22925312 -> 22929408 00010000:00010000:3.0:1789760728.735529:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.735542:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f35 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.735551:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f35 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.735558:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760728.735563:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f35 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.735571:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f35 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.735602:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f35 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.735610:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f35 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760728.735823:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493697920:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48355us (48660us total) trans 4294972916 rc 0/0 00000100:00100000:3.0:1789760728.735835:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5627 00000100:00100000:3.0:1789760728.735844:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738967159936 time 739028391726 state 0x2 ref 0 00000100:00100000:2.0:1789760728.740489:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760728.740695:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493698048 00000100:00100000:0.0:1789760728.740704:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493698048 00000100:00100000:0.0:1789760728.740756:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 738967159936 time 739028391726 state 0x0 ref 1 00000100:00100000:0.0:1789760728.740771:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739028652142 time 739028391726 state 0x1 ref 1 00000100:00100000:0.0:1789760728.740846:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739028652142 time 739081881725 state 0x0 ref 1 00000100:00100000:0.0:1789760728.740861:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493698048:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760728.740909:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22929408, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:0.0:1789760728.740918:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e04f3c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.740934:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760728.740939:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.740945:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e04f3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.740996:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e04f3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.779604:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972917 is committed 00002000:00010000:0.0:1789760728.779847:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22929408 -> 22933504 00010000:00010000:0.0:1789760728.779864:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e04f3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.779877:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e04f3c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.779886:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e04f3c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.779893:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760728.779897:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e04f3c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.779906:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e04f3c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.779925:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e04f3c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.779933:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e04f3c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.780055:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493698048:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39190us (39570us total) trans 4294972917 rc 0/0 00000100:00100000:0.0:1789760728.780065:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5628 00000100:00100000:0.0:1789760728.780074:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739028652142 time 739081881725 state 0x2 ref 0 00000100:00100000:0.0:1789760728.785278:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760728.785357:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493698176 00000100:00100000:1.0:1789760728.785431:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493698176 00000100:00100000:1.0:1789760728.785505:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739028652142 time 739081881725 state 0x0 ref 1 00000100:00100000:1.0:1789760728.785520:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739082142141 time 739081881725 state 0x1 ref 1 00000100:00100000:1.0:1789760728.785554:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739082142141 time 739126589336 state 0x0 ref 1 00000100:00100000:1.0:1789760728.785567:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493698176:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760728.785623:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22933504, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:1.0:1789760728.785641:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f43 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.785657:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760728.785662:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.785668:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.785680:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760728.823708:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972918 is committed 00002000:00010000:1.0:1789760728.823855:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22933504 -> 22937600 00010000:00010000:1.0:1789760728.823871:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.823883:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f43 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.823891:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f43 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.823898:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760728.823903:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f43 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.823913:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f43 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.823933:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f43 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.823940:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f43 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.824182:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493698176:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38612us (38909us total) trans 4294972918 rc 0/0 00000100:00100000:1.0:1789760728.824195:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5629 00000100:00100000:1.0:1789760728.824204:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739082142141 time 739126589336 state 0x2 ref 0 00000100:00100000:0.0:1789760728.828255:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760728.828346:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493698304 00000100:00100000:1.0:1789760728.828356:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493698304 00000100:00100000:1.0:1789760728.828409:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739082142141 time 739126589336 state 0x0 ref 1 00000100:00100000:1.0:1789760728.828425:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739126849752 time 739126589336 state 0x1 ref 1 00000100:00100000:1.0:1789760728.828487:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739126849752 time 739169522109 state 0x0 ref 1 00000100:00100000:1.0:1789760728.828498:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493698304:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760728.828543:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22937600, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:1.0:1789760728.828551:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f4a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.828566:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760728.828578:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.828586:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.828599:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.859666:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972919 is committed 00002000:00010000:1.0:1789760728.859822:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22937600 -> 22941696 00010000:00010000:1.0:1789760728.859837:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.859849:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f4a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.859858:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.859865:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760728.859870:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.859879:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f4a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.859902:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f4a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.859909:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f4a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.860026:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493698304:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31526us (31776us total) trans 4294972919 rc 0/0 00000100:00100000:1.0:1789760728.860036:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5630 00000100:00100000:1.0:1789760728.860045:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739126849752 time 739169522109 state 0x2 ref 0 00000100:00100000:0.0:1789760728.865754:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760728.865848:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493698432 00000100:00100000:2.0:1789760728.865860:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493698432 00000100:00100000:2.0:1789760728.865911:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739126849752 time 739169522109 state 0x0 ref 1 00000100:00100000:2.0:1789760728.865925:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739169782525 time 739169522109 state 0x1 ref 1 00000100:00100000:2.0:1789760728.865949:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739169782525 time 739206985630 state 0x0 ref 1 00000100:00100000:2.0:1789760728.865975:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493698432:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760728.866023:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22941696, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:2.0:1789760728.866032:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04f51 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.866047:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760728.866225:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760728.866233:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04f51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.866248:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04f51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760728.907422:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972920 is committed 00002000:00010000:2.0:1789760728.907486:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22941696 -> 22945792 00010000:00010000:2.0:1789760728.907499:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04f51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.907511:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04f51 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.907519:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04f51 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.907532:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760728.907538:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04f51 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.907547:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04f51 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.907565:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04f51 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.907582:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04f51 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760728.907687:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493698432:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41720us (41935us total) trans 4294972920 rc 0/0 00000100:00100000:2.0:1789760728.907697:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5631 00000100:00100000:2.0:1789760728.907707:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739169782525 time 739206985630 state 0x2 ref 0 00000100:00100000:0.0:1789760728.916894:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760728.917140:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493698560 00000100:00100000:2.0:1789760728.917151:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493698560 00000100:00100000:2.0:1789760728.917282:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739169782525 time 739206985630 state 0x0 ref 1 00000100:00100000:2.0:1789760728.917298:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739207246046 time 739206985630 state 0x1 ref 1 00000100:00100000:2.0:1789760728.917323:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739207246046 time 739258359044 state 0x0 ref 1 00000100:00100000:2.0:1789760728.917339:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493698560:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760728.917391:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22945792, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:2.0:1789760728.917400:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04f58 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.917416:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760728.917421:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760728.917427:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04f58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.917441:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04f58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760728.956468:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc48de680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3303:1876699519250816:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760728.956544:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760728.956820:0:6691:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519250816 00000100:00100000:2.0:1789760728.956828:0:6691:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519250816 00000100:00100000:2.0:1789760728.956873:0:6691:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 195 00000100:00100000:2.0:1789760728.956883:0:6691:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876699519250816:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760728.957006:0:6691:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760728.957057:0:6691:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876699519250816:12345-0@lo:13:osp-pre-1-0.0 Request processed in 174us (512us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760728.957064:0:6691:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 195 00000100:00100000:2.0:1789760728.957292:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc48de680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3303:1876699519250816:0@lo:13:osp-pre-1-0.0 00000001:00080000:0.0:1789760728.968031:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972921 is committed 00002000:00010000:3.0:1789760728.968156:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22945792 -> 22949888 00010000:00010000:3.0:1789760728.968170:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04f58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.968184:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04f58 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.968192:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04f58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.968199:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760728.968204:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04f58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.968212:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04f58 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.968230:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04f58 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.968240:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e04f58 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760728.968332:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493698560:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50991us (51443us total) trans 4294972921 rc 0/0 00000100:00100000:3.0:1789760728.968342:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5632 00000100:00100000:3.0:1789760728.968352:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739207246046 time 739258359044 state 0x2 ref 0 00000100:00100000:0.0:1789760728.972849:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760728.973313:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493698688 00000100:00100000:1.0:1789760728.973325:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493698688 00000100:00100000:1.0:1789760728.973388:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739207246046 time 739258359044 state 0x0 ref 1 00000100:00100000:1.0:1789760728.973403:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739258619460 time 739258359044 state 0x1 ref 1 00000100:00100000:1.0:1789760728.973433:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739258619460 time 739314468579 state 0x0 ref 1 00000100:00100000:1.0:1789760728.973445:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493698688:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760728.973493:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22949888, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:1.0:1789760728.973503:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f5f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.973519:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760728.973524:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.973531:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.973544:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760729.028422:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22949888 -> 22953984 00010000:00010000:1.0:1789760729.028442:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.028455:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f5f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.028464:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.028470:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760729.028476:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.028485:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f5f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.028503:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f5f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.028510:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f5f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.028612:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493698688:12345-192.168.203.60@tcp:4:dd.0 Request processed in 55163us (55766us total) trans 4294972922 rc 0/0 00000100:00100000:1.0:1789760729.028629:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5633 00000100:00100000:1.0:1789760729.028639:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739258619460 time 739314468579 state 0x2 ref 0 00000001:00080000:3.0:1789760729.028846:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972922 is committed 00000100:00100000:0.0:1789760729.035914:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760729.037515:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493698816 00000100:00100000:1.0:1789760729.037528:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493698816 00000100:00100000:1.0:1789760729.037662:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739258619460 time 739314468579 state 0x0 ref 1 00000100:00100000:1.0:1789760729.037679:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739314728995 time 739314468579 state 0x1 ref 1 00000100:00100000:1.0:1789760729.037710:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739314728995 time 739378746171 state 0x0 ref 1 00000100:00100000:1.0:1789760729.037726:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493698816:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760729.037776:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22953984, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:1.0:1789760729.037784:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f66 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.037799:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760729.037803:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.037809:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.037822:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760729.038418:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789760729.074368:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22953984 -> 22958080 00010000:00010000:3.0:1789760729.074389:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.074404:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f66 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.074413:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f66 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.074421:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760729.074426:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f66 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.074435:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f66 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.074459:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f66 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.074468:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e04f66 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.074560:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493698816:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36833us (38651us total) trans 4294972923 rc 0/0 00000100:00100000:3.0:1789760729.074579:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5634 00000100:00100000:3.0:1789760729.074589:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739314728995 time 739378746171 state 0x2 ref 0 00000001:00080000:3.0:1789760729.074638:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972923 is committed 00000100:00100000:0.0:1789760729.082032:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760729.082241:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493698944 00000100:00100000:2.0:1789760729.082253:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493698944 00000100:00100000:2.0:1789760729.082362:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739314728995 time 739378746171 state 0x0 ref 1 00000100:00100000:2.0:1789760729.082378:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739379006587 time 739378746171 state 0x1 ref 1 00000100:00100000:2.0:1789760729.082406:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739379006587 time 739423441445 state 0x0 ref 1 00000100:00100000:2.0:1789760729.082418:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493698944:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760729.082466:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22958080, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:2.0:1789760729.082476:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04f6d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.082492:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760729.082497:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.082503:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04f6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.082518:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04f6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.121033:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972924 is committed 00002000:00010000:3.0:1789760729.121392:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22958080 -> 22962176 00010000:00010000:3.0:1789760729.121407:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04f6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.121420:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04f6d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.121428:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04f6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.121435:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760729.121439:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04f6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.121453:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04f6d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.121474:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04f6d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.121482:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e04f6d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.121620:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493698944:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39199us (39593us total) trans 4294972924 rc 0/0 00000100:00100000:3.0:1789760729.121681:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5635 00000100:00100000:3.0:1789760729.121691:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739379006587 time 739423441445 state 0x2 ref 0 00000100:00100000:0.0:1789760729.125829:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760729.127744:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493699072 00000100:00100000:3.0:1789760729.127758:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493699072 00000100:00100000:3.0:1789760729.127829:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739379006587 time 739423441445 state 0x0 ref 1 00000100:00100000:3.0:1789760729.128038:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739423701861 time 739423441445 state 0x1 ref 1 00000100:00100000:3.0:1789760729.130014:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739423701861 time 739471037590 state 0x0 ref 1 00000100:00100000:3.0:1789760729.130043:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493699072:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760729.131746:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22962176, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:3.0:1789760729.131762:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f74 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.131778:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760729.131782:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.131786:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.131797:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.170347:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972925 is committed 00002000:00010000:3.0:1789760729.170506:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22962176 -> 22966272 00010000:00010000:3.0:1789760729.170523:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.170537:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f74 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.170547:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.170554:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760729.170559:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.170579:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f74 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.170718:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f74 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.170729:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f74 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.170876:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493699072:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40839us (45049us total) trans 4294972925 rc 0/0 00000100:00100000:3.0:1789760729.170887:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5636 00000100:00100000:3.0:1789760729.170897:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739423701861 time 739471037590 state 0x2 ref 0 00000100:00100000:0.0:1789760729.175536:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760729.175828:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493699200 00000100:00100000:1.0:1789760729.175843:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493699200 00000100:00100000:1.0:1789760729.175907:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739423701861 time 739471037590 state 0x0 ref 1 00000100:00100000:1.0:1789760729.175922:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739471298006 time 739471037590 state 0x1 ref 1 00000100:00100000:1.0:1789760729.175955:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739471298006 time 739516989852 state 0x0 ref 1 00000100:00100000:1.0:1789760729.176004:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493699200:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760729.176063:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22966272, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:1.0:1789760729.176072:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04f7b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.176089:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760729.176093:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.176100:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04f7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.176115:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04f7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.219401:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972926 is committed 00002000:00010000:1.0:1789760729.220204:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22966272 -> 22970368 00010000:00010000:1.0:1789760729.220221:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04f7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.220236:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04f7b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.220246:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04f7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.220254:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760729.220260:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04f7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.220270:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04f7b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.220291:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04f7b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.220299:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04f7b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.220445:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493699200:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44439us (44914us total) trans 4294972926 rc 0/0 00000100:00100000:1.0:1789760729.220455:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5637 00000100:00100000:1.0:1789760729.220464:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739471298006 time 739516989852 state 0x2 ref 0 00000100:00100000:0.0:1789760729.225938:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760729.226101:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493699328 00000100:00100000:1.0:1789760729.226111:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493699328 00000100:00100000:1.0:1789760729.226166:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739471298006 time 739516989852 state 0x0 ref 1 00000100:00100000:1.0:1789760729.226182:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739517250268 time 739516989852 state 0x1 ref 1 00000100:00100000:1.0:1789760729.226214:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739517250268 time 739567249678 state 0x0 ref 1 00000100:00100000:1.0:1789760729.226226:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493699328:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760729.226279:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22970368, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:1.0:1789760729.226289:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04f82 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.226304:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760729.226309:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.226316:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04f82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.226329:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04f82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760729.295031:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972927 is committed 00002000:00010000:3.0:1789760729.295149:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22970368 -> 22974464 00010000:00010000:3.0:1789760729.295165:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04f82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.295178:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04f82 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.295187:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04f82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.295195:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760729.295200:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04f82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.295209:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04f82 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.295243:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04f82 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.295250:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e04f82 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.295489:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493699328:12345-192.168.203.60@tcp:4:dd.0 Request processed in 69257us (69557us total) trans 4294972927 rc 0/0 00000100:00100000:3.0:1789760729.295501:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5638 00000100:00100000:3.0:1789760729.295512:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739517250268 time 739567249678 state 0x2 ref 0 00000100:00100000:0.0:1789760729.301629:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760729.304373:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493699456 00000100:00100000:3.0:1789760729.304388:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493699456 00000100:00100000:3.0:1789760729.304450:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739517250268 time 739567249678 state 0x0 ref 1 00000100:00100000:3.0:1789760729.304465:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739567510094 time 739567249678 state 0x1 ref 1 00000100:00100000:3.0:1789760729.304501:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739567510094 time 739645537227 state 0x0 ref 1 00000100:00100000:3.0:1789760729.304515:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493699456:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760729.304563:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22974464, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:3.0:1789760729.304582:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f89 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.304598:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760729.304603:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.304609:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.304622:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760729.330005:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972928 is committed 00002000:00010000:3.0:1789760729.330200:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22974464 -> 22978560 00010000:00010000:3.0:1789760729.330215:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.330229:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f89 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.330237:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.330245:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760729.330249:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.330258:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f89 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.330277:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f89 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.330290:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04f89 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.330405:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493699456:12345-192.168.203.60@tcp:4:dd.0 Request processed in 25888us (28780us total) trans 4294972928 rc 0/0 00000100:00100000:3.0:1789760729.330415:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5639 00000100:00100000:3.0:1789760729.330424:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739567510094 time 739645537227 state 0x2 ref 0 00000100:00100000:0.0:1789760729.334489:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760729.334566:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493699584 00000100:00100000:3.0:1789760729.334583:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493699584 00000100:00100000:3.0:1789760729.334687:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739567510094 time 739645537227 state 0x0 ref 1 00000100:00100000:3.0:1789760729.334703:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739645797643 time 739645537227 state 0x1 ref 1 00000100:00100000:3.0:1789760729.334831:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739645797643 time 739675865593 state 0x0 ref 1 00000100:00100000:3.0:1789760729.334846:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493699584:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760729.334895:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22978560, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:3.0:1789760729.334903:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04f90 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.334920:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760729.334924:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.334930:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04f90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.334949:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04f90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760729.374109:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972929 is committed 00002000:00010000:3.0:1789760729.374227:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22978560 -> 22982656 00010000:00010000:3.0:1789760729.374247:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04f90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.374267:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04f90 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.374278:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04f90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.374286:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760729.374292:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04f90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.374313:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04f90 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.374339:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04f90 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.374348:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04f90 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.374479:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493699584:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39629us (39993us total) trans 4294972929 rc 0/0 00000100:00100000:3.0:1789760729.374491:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5640 00000100:00100000:3.0:1789760729.374505:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739645797643 time 739675865593 state 0x2 ref 0 00000100:00100000:0.0:1789760729.380750:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760729.381214:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493699712 00000100:00100000:1.0:1789760729.381226:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493699712 00000100:00100000:2.0:1789760729.381374:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739645797643 time 739675865593 state 0x0 ref 1 00000100:00100000:2.0:1789760729.381390:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739676126009 time 739675865593 state 0x1 ref 1 00000100:00100000:2.0:1789760729.381423:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739676126009 time 739722458615 state 0x0 ref 1 00000100:00100000:2.0:1789760729.381440:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493699712:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760729.381488:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22982656, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:2.0:1789760729.381497:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04f97 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.381513:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760729.381517:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.381523:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04f97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.381538:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04f97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760729.408935:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22982656 -> 22986752 00010000:00010000:2.0:1789760729.408954:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04f97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.408988:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04f97 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.408997:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04f97 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.409005:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760729.409011:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04f97 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.409020:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04f97 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.409040:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04f97 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.409057:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04f97 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.409153:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493699712:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27711us (28409us total) trans 4294972930 rc 0/0 00000100:00100000:2.0:1789760729.409164:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5641 00000100:00100000:2.0:1789760729.409173:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739676126009 time 739722458615 state 0x2 ref 0 00000001:00080000:0.0:1789760729.413105:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972930 is committed 00000100:00100000:2.0:1789760729.413403:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760729.413550:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493699840 00000100:00100000:3.0:1789760729.413559:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493699840 00000100:00100000:3.0:1789760729.413622:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739676126009 time 739722458615 state 0x0 ref 1 00000100:00100000:3.0:1789760729.413804:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739722719031 time 739722458615 state 0x1 ref 1 00000100:00100000:3.0:1789760729.413862:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739722719031 time 739754897426 state 0x0 ref 1 00000100:00100000:3.0:1789760729.413875:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493699840:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760729.413926:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22986752, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:3.0:1789760729.413936:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04f9e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.413952:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760729.413956:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.413983:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04f9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.413998:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04f9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760729.445353:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972931 is committed 00002000:00010000:2.0:1789760729.445504:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22986752 -> 22990848 00010000:00010000:2.0:1789760729.445519:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04f9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.445531:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04f9e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.445542:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04f9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.445555:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760729.445560:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04f9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.445585:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04f9e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.445604:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04f9e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.445612:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e04f9e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.445706:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493699840:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31830us (32304us total) trans 4294972931 rc 0/0 00000100:00100000:2.0:1789760729.445716:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5642 00000100:00100000:2.0:1789760729.445726:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739722719031 time 739754897426 state 0x2 ref 0 00000100:00100000:0.0:1789760729.448865:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760729.449034:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493699968 00000100:00100000:3.0:1789760729.449115:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493699968 00000100:00100000:3.0:1789760729.449181:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739722719031 time 739754897426 state 0x0 ref 1 00000100:00100000:3.0:1789760729.449198:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739755157842 time 739754897426 state 0x1 ref 1 00000100:00100000:3.0:1789760729.449265:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739755157842 time 739790300017 state 0x0 ref 1 00000100:00100000:3.0:1789760729.449279:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493699968:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760729.449339:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22990848, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:3.0:1789760729.449348:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04fa5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.449365:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760729.449369:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.449376:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04fa5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.449390:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04fa5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.486610:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972932 is committed 00002000:00010000:0.0:1789760729.486686:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22990848 -> 22994944 00010000:00010000:0.0:1789760729.486702:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04fa5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.486716:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04fa5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.486726:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04fa5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.486733:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760729.486738:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04fa5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.486748:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04fa5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.486768:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04fa5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.486782:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e04fa5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.486900:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493699968:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37619us (38040us total) trans 4294972932 rc 0/0 00000100:00100000:0.0:1789760729.486910:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5643 00000100:00100000:0.0:1789760729.486919:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739755157842 time 739790300017 state 0x2 ref 0 00000100:00100000:0.0:1789760729.493253:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760729.493502:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493700096 00000100:00100000:2.0:1789760729.493514:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493700096 00000100:00100000:2.0:1789760729.493579:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739755157842 time 739790300017 state 0x0 ref 1 00000100:00100000:2.0:1789760729.493595:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739790560433 time 739790300017 state 0x1 ref 1 00000100:00100000:2.0:1789760729.493719:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739790560433 time 739834753706 state 0x0 ref 1 00000100:00100000:2.0:1789760729.493739:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493700096:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760729.493788:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22994944, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:2.0:1789760729.493797:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04fac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.493813:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760729.493818:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.493824:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04fac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.493838:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04fac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.517992:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972933 is committed 00002000:00010000:0.0:1789760729.518086:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22994944 -> 22999040 00010000:00010000:0.0:1789760729.518102:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04fac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.518115:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04fac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.518124:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04fac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.518132:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760729.518137:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04fac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.518146:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04fac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.518171:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04fac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.518179:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e04fac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.518309:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493700096:12345-192.168.203.60@tcp:4:dd.0 Request processed in 24569us (25061us total) trans 4294972933 rc 0/0 00000100:00100000:0.0:1789760729.518320:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5644 00000100:00100000:0.0:1789760729.518331:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739790560433 time 739834753706 state 0x2 ref 0 00000100:00100000:3.0:1789760729.523723:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760729.524085:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493700224 00000100:00100000:2.0:1789760729.524095:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493700224 00000100:00100000:2.0:1789760729.524150:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739790560433 time 739834753706 state 0x0 ref 1 00000100:00100000:2.0:1789760729.524164:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739835014122 time 739834753706 state 0x1 ref 1 00000100:00100000:2.0:1789760729.524233:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739835014122 time 739865267855 state 0x0 ref 1 00000100:00100000:2.0:1789760729.524246:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493700224:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760729.524293:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 22999040, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:2.0:1789760729.524301:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04fb3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.524322:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760729.524327:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.524333:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04fb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.524345:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04fb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760729.560084:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 22999040 -> 23003136 00010000:00010000:0.0:1789760729.560105:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04fb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.560119:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04fb3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.560129:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04fb3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.560138:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760729.560143:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04fb3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.560160:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04fb3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.560179:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04fb3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.560188:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e04fb3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.560283:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493700224:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36035us (36565us total) trans 4294972934 rc 0/0 00000100:00100000:0.0:1789760729.560294:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5645 00000100:00100000:0.0:1789760729.560304:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739835014122 time 739865267855 state 0x2 ref 0 00000001:00080000:0.0:1789760729.561048:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972934 is committed 00000100:00100000:1.0:1789760729.564345:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760729.564882:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493700352 00000100:00100000:3.0:1789760729.564892:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493700352 00000100:00100000:3.0:1789760729.565142:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739835014122 time 739865267855 state 0x0 ref 1 00000100:00100000:3.0:1789760729.565160:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739865528271 time 739865267855 state 0x1 ref 1 00000100:00100000:3.0:1789760729.565219:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739865528271 time 739906253302 state 0x0 ref 1 00000100:00100000:3.0:1789760729.565232:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493700352:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760729.565278:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23003136, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:3.0:1789760729.565285:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04fba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.565297:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760729.565305:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.565309:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04fba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.565320:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04fba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.608507:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972935 is committed 00002000:00010000:0.0:1789760729.608834:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23003136 -> 23007232 00010000:00010000:0.0:1789760729.608851:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04fba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.608866:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04fba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.608876:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04fba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.608889:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760729.608895:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04fba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.608905:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04fba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.608925:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04fba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.608934:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e04fba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.610027:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493700352:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44788us (45691us total) trans 4294972935 rc 0/0 00000100:00100000:0.0:1789760729.610040:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5646 00000100:00100000:0.0:1789760729.610060:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739865528271 time 739906253302 state 0x2 ref 0 00000100:00100000:2.0:1789760729.614316:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760729.614468:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493700480 00000100:00100000:3.0:1789760729.614478:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493700480 00000100:00100000:3.0:1789760729.614533:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739865528271 time 739906253302 state 0x0 ref 1 00000100:00100000:3.0:1789760729.614548:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739906513718 time 739906253302 state 0x1 ref 1 00000100:00100000:3.0:1789760729.614621:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739906513718 time 739955655822 state 0x0 ref 1 00000100:00100000:3.0:1789760729.614635:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493700480:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760729.614686:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23007232, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0xb007 00010000:00010000:3.0:1789760729.614695:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04fc1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.614710:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760729.614714:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.614720:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04fc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.614733:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04fc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.658094:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760729.660282:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972936 is committed 00002000:00010000:2.0:1789760729.660760:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23007232 -> 23011328 00002000:00010000:2.0:1789760729.660765:0:13038:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760728 -> 1789760729 00002000:00010000:2.0:1789760729.660768:0:13038:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760728 -> 1789760729 00010000:00010000:2.0:1789760729.660780:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04fc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.660793:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04fc1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.660803:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04fc1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.660811:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760729.660815:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04fc1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.660823:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04fc1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.660845:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04fc1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.660853:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e04fc1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.660942:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493700480:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46303us (46630us total) trans 4294972936 rc 0/0 00000100:00100000:2.0:1789760729.660952:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5647 00000100:00100000:2.0:1789760729.660990:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739906513718 time 739955655822 state 0x2 ref 0 00000100:00100000:3.0:1789760729.673087:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760729.673266:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493700608 00000100:00100000:0.0:1789760729.673277:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493700608 00000100:00100000:0.0:1789760729.673327:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739906513718 time 739955655822 state 0x0 ref 1 00000100:00100000:0.0:1789760729.673341:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739955916238 time 739955655822 state 0x1 ref 1 00000100:00100000:0.0:1789760729.673371:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739955916238 time 740014407006 state 0x0 ref 1 00000100:00100000:0.0:1789760729.673384:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493700608:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760729.673431:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23011328, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:0.0:1789760729.673440:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04fc8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.673456:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760729.673461:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.673467:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04fc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.673481:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04fc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.707730:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972937 is committed 00002000:00010000:1.0:1789760729.707840:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23011328 -> 23015424 00010000:00010000:1.0:1789760729.707857:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04fc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.707870:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04fc8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.707878:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04fc8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.707884:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760729.707889:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04fc8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.707897:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04fc8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.707916:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04fc8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.707924:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e04fc8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.708134:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493700608:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34746us (35052us total) trans 4294972937 rc 0/0 00000100:00100000:1.0:1789760729.708145:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5648 00000100:00100000:1.0:1789760729.708156:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739955916238 time 740014407006 state 0x2 ref 0 00000100:00100000:1.0:1789760729.716068:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760729.716156:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493700736 00000100:00100000:0.0:1789760729.716166:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493700736 00000100:00100000:0.0:1789760729.716227:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 739955916238 time 740014407006 state 0x0 ref 1 00000100:00100000:0.0:1789760729.716242:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740014667422 time 740014407006 state 0x1 ref 1 00000100:00100000:0.0:1789760729.716279:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740014667422 time 740057314322 state 0x0 ref 1 00000100:00100000:0.0:1789760729.716292:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493700736:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760729.716343:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23015424, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:0.0:1789760729.716353:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04fcf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.716375:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760729.716379:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.716386:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04fcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.716401:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04fcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.777756:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972938 is committed 00002000:00010000:1.0:1789760729.777998:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23015424 -> 23019520 00010000:00010000:1.0:1789760729.778014:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04fcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.778027:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04fcf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.778037:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04fcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.778045:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760729.778049:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04fcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.778058:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04fcf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.778077:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04fcf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.778084:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e04fcf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.778243:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493700736:12345-192.168.203.60@tcp:4:dd.0 Request processed in 61947us (62179us total) trans 4294972938 rc 0/0 00000100:00100000:1.0:1789760729.778258:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5649 00000100:00100000:1.0:1789760729.778267:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740014667422 time 740057314322 state 0x2 ref 0 00000100:00100000:2.0:1789760729.784167:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760729.784351:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493700864 00000100:00100000:3.0:1789760729.784361:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493700864 00000100:00100000:3.0:1789760729.784413:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740014667422 time 740057314322 state 0x0 ref 1 00000100:00100000:3.0:1789760729.784428:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740057574738 time 740057314322 state 0x1 ref 1 00000100:00100000:3.0:1789760729.784816:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740057574738 time 740125849681 state 0x0 ref 1 00000100:00100000:3.0:1789760729.784832:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493700864:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760729.784882:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23019520, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:3.0:1789760729.784891:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04fd6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.784908:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760729.785302:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.785310:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04fd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.785324:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04fd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.849244:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972939 is committed 00002000:00010000:0.0:1789760729.849368:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23019520 -> 23023616 00010000:00010000:0.0:1789760729.849384:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04fd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.849396:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04fd6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.849406:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04fd6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.849414:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760729.849419:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04fd6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.849429:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04fd6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.849449:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04fd6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.849459:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e04fd6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.849558:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493700864:12345-192.168.203.60@tcp:4:dd.0 Request processed in 64725us (65720us total) trans 4294972939 rc 0/0 00000100:00100000:0.0:1789760729.849579:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5650 00000100:00100000:0.0:1789760729.849590:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740057574738 time 740125849681 state 0x2 ref 0 00000100:00100000:3.0:1789760729.854366:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760729.855186:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493700992 00000100:00100000:2.0:1789760729.855197:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493700992 00000100:00100000:2.0:1789760729.855254:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740057574738 time 740125849681 state 0x0 ref 1 00000100:00100000:2.0:1789760729.855271:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740126110097 time 740125849681 state 0x1 ref 1 00000100:00100000:2.0:1789760729.855307:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740126110097 time 740196342695 state 0x0 ref 1 00000100:00100000:2.0:1789760729.855321:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493700992:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760729.855370:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23023616, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:2.0:1789760729.855380:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04fdd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.855401:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760729.855406:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.855412:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04fdd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.855428:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04fdd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.917385:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972940 is committed 00002000:00010000:2.0:1789760729.917552:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23023616 -> 23027712 00010000:00010000:2.0:1789760729.917576:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04fdd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.917588:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04fdd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.917597:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04fdd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.917605:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760729.917610:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04fdd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.917998:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04fdd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.918020:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04fdd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.918028:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04fdd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.918122:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493700992:12345-192.168.203.60@tcp:4:dd.0 Request processed in 62799us (63770us total) trans 4294972940 rc 0/0 00000100:00100000:2.0:1789760729.918133:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5651 00000100:00100000:2.0:1789760729.918142:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740126110097 time 740196342695 state 0x2 ref 0 00000100:00100000:1.0:1789760729.923873:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760729.924018:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493701120 00000100:00100000:0.0:1789760729.924028:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493701120 00000100:00100000:0.0:1789760729.924098:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740126110097 time 740196342695 state 0x0 ref 1 00000100:00100000:0.0:1789760729.924114:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740196603111 time 740196342695 state 0x1 ref 1 00000100:00100000:0.0:1789760729.924142:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740196603111 time 740265178269 state 0x0 ref 1 00000100:00100000:0.0:1789760729.924155:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493701120:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760729.924204:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23027712, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:0.0:1789760729.924213:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04fe4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.924230:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760729.924234:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.924241:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04fe4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.924261:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04fe4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760729.991550:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23027712 -> 23031808 00010000:00010000:2.0:1789760729.991580:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04fe4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.991594:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04fe4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.991604:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04fe4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.991613:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760729.991626:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04fe4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.991636:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04fe4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.991669:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04fe4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.991677:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e04fe4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.991774:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493701120:12345-192.168.203.60@tcp:4:dd.0 Request processed in 67615us (67906us total) trans 4294972941 rc 0/0 00000100:00100000:2.0:1789760729.991784:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5652 00000100:00100000:2.0:1789760729.991795:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740196603111 time 740265178269 state 0x2 ref 0 00000001:00080000:3.0:1789760729.992273:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972941 is committed 00000100:00100000:3.0:1789760729.997302:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760729.997541:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493701248 00000100:00100000:1.0:1789760729.997551:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493701248 00000100:00100000:1.0:1789760729.997616:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740196603111 time 740265178269 state 0x0 ref 1 00000100:00100000:1.0:1789760729.997631:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740265438685 time 740265178269 state 0x1 ref 1 00000100:00100000:1.0:1789760729.997707:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740265438685 time 740338741589 state 0x0 ref 1 00000100:00100000:1.0:1789760729.997721:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493701248:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760729.997767:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23031808, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:1.0:1789760729.997777:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04feb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.997792:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760729.997796:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.997802:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04feb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.997816:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04feb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760730.050203:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23031808 -> 23035904 00010000:00010000:1.0:1789760730.050234:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04feb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.050247:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04feb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.050256:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04feb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.050263:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760730.050268:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04feb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.050277:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04feb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.050297:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04feb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.050304:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e04feb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.050399:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493701248:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52675us (53100us total) trans 4294972942 rc 0/0 00000100:00100000:1.0:1789760730.050413:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5653 00000100:00100000:1.0:1789760730.050422:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740265438685 time 740338741589 state 0x2 ref 0 00000001:00080000:3.0:1789760730.051333:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972942 is committed 00000100:00100000:0.0:1789760730.061275:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760730.063356:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493701376 00000100:00100000:3.0:1789760730.063372:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493701376 00000100:00100000:3.0:1789760730.063437:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740265438685 time 740338741589 state 0x0 ref 1 00000100:00100000:3.0:1789760730.063453:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740339002005 time 740338741589 state 0x1 ref 1 00000100:00100000:3.0:1789760730.063486:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740339002005 time 740404521908 state 0x0 ref 1 00000100:00100000:3.0:1789760730.063499:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493701376:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760730.063664:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23035904, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:3.0:1789760730.063674:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04ff2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.063695:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760730.063700:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760730.063706:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04ff2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.063720:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04ff2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760730.137274:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23035904 -> 23040000 00010000:00010000:1.0:1789760730.137295:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04ff2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.137309:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04ff2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760730.137316:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972943 is committed 00010000:00010000:1.0:1789760730.137318:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04ff2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.137340:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760730.137345:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04ff2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.137355:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04ff2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.137374:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04ff2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.137382:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e04ff2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.137486:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493701376:12345-192.168.203.60@tcp:4:dd.0 Request processed in 73985us (76216us total) trans 4294972943 rc 0/0 00000100:00100000:1.0:1789760730.137496:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5654 00000100:00100000:1.0:1789760730.137506:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740339002005 time 740404521908 state 0x2 ref 0 00000100:00100000:0.0:1789760730.146839:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760730.147019:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493701504 00000100:00100000:2.0:1789760730.147028:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493701504 00000100:00100000:2.0:1789760730.147100:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740339002005 time 740404521908 state 0x0 ref 1 00000100:00100000:2.0:1789760730.147117:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740404782324 time 740404521908 state 0x1 ref 1 00000100:00100000:2.0:1789760730.147147:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740404782324 time 740488182844 state 0x0 ref 1 00000100:00100000:2.0:1789760730.147158:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493701504:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760730.147205:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23040000, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:2.0:1789760730.147215:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04ff9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.147231:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760730.147236:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.147243:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04ff9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.147258:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04ff9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760730.217234:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972944 is committed 00002000:00010000:0.0:1789760730.217332:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23040000 -> 23044096 00010000:00010000:0.0:1789760730.217347:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04ff9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.217360:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04ff9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.217370:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04ff9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.217378:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760730.217383:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04ff9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.217392:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04ff9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.217411:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04ff9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.217424:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e04ff9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.217518:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493701504:12345-192.168.203.60@tcp:4:dd.0 Request processed in 70357us (70686us total) trans 4294972944 rc 0/0 00000100:00100000:0.0:1789760730.217529:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5655 00000100:00100000:0.0:1789760730.217538:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740404782324 time 740488182844 state 0x2 ref 0 00000100:00100000:3.0:1789760730.221359:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760730.221479:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493701632 00000100:00100000:0.0:1789760730.221489:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493701632 00000100:00100000:0.0:1789760730.221553:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740404782324 time 740488182844 state 0x0 ref 1 00000100:00100000:0.0:1789760730.221576:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740488443260 time 740488182844 state 0x1 ref 1 00000100:00100000:0.0:1789760730.221597:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740488443260 time 740562633092 state 0x0 ref 1 00000100:00100000:0.0:1789760730.221610:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493701632:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760730.221914:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23044096, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:0.0:1789760730.221924:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05000 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.221939:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760730.221943:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.221949:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05000 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.222313:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05000 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760730.255759:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972945 is committed 00002000:00010000:1.0:1789760730.255846:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23044096 -> 23048192 00010000:00010000:1.0:1789760730.255862:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05000 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.255875:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05000 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.255884:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05000 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.255892:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760730.255897:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05000 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.255906:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05000 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.255925:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05000 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.255938:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05000 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.256073:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493701632:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34460us (34718us total) trans 4294972945 rc 0/0 00000100:00100000:1.0:1789760730.256084:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5656 00000100:00100000:1.0:1789760730.256094:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740488443260 time 740562633092 state 0x2 ref 0 00000100:00100000:3.0:1789760730.259151:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760730.259406:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493701760 00000100:00100000:1.0:1789760730.259416:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493701760 00000100:00100000:1.0:1789760730.259471:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740488443260 time 740562633092 state 0x0 ref 1 00000100:00100000:1.0:1789760730.259487:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740562893508 time 740562633092 state 0x1 ref 1 00000100:00100000:1.0:1789760730.259582:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740562893508 time 740600616881 state 0x0 ref 1 00000100:00100000:1.0:1789760730.259596:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493701760:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760730.259644:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23048192, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:1.0:1789760730.259652:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05007 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.259668:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760730.259672:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.259679:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05007 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.259698:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05007 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760730.302182:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23048192 -> 23052288 00000001:00080000:3.0:1789760730.302185:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972946 is committed 00010000:00010000:2.0:1789760730.302200:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05007 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.302213:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05007 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.302222:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05007 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.302230:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760730.302233:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05007 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.302239:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05007 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.302254:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05007 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.302263:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05007 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.302347:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493701760:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42749us (43200us total) trans 4294972946 rc 0/0 00000100:00100000:2.0:1789760730.302357:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5657 00000100:00100000:2.0:1789760730.302368:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740562893508 time 740600616881 state 0x2 ref 0 00000100:00100000:2.0:1789760730.310519:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760730.310736:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493701888 00000100:00100000:1.0:1789760730.310746:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493701888 00000100:00100000:1.0:1789760730.310809:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740562893508 time 740600616881 state 0x0 ref 1 00000100:00100000:1.0:1789760730.310825:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740600877297 time 740600616881 state 0x1 ref 1 00000100:00100000:1.0:1789760730.310850:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740600877297 time 740651885802 state 0x0 ref 1 00000100:00100000:1.0:1789760730.310864:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493701888:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760730.310920:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23052288, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:1.0:1789760730.310940:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0500e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.310956:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760730.311006:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.311013:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0500e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.311024:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0500e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760730.348371:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23052288 -> 23056384 00010000:00010000:1.0:1789760730.348389:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0500e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.348403:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0500e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.348421:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0500e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.348429:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760730.348433:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0500e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.348442:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0500e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.348461:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0500e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.348468:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0500e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.348555:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493701888:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37690us (38053us total) trans 4294972947 rc 0/0 00000100:00100000:1.0:1789760730.348573:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5658 00000100:00100000:1.0:1789760730.348583:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740600877297 time 740651885802 state 0x2 ref 0 00000001:00080000:3.0:1789760730.350466:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972947 is committed 00000100:00100000:3.0:1789760730.353421:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760730.353611:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493702016 00000100:00100000:0.0:1789760730.353629:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493702016 00000100:00100000:0.0:1789760730.353697:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740600877297 time 740651885802 state 0x0 ref 1 00000100:00100000:0.0:1789760730.353712:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740652146218 time 740651885802 state 0x1 ref 1 00000100:00100000:0.0:1789760730.353743:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740652146218 time 740694778389 state 0x0 ref 1 00000100:00100000:0.0:1789760730.353756:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493702016:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760730.353804:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23056384, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:0.0:1789760730.353813:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05015 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.353828:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760730.353833:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.353839:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05015 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.353852:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05015 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760730.393063:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23056384 -> 23060480 00010000:00010000:1.0:1789760730.393083:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05015 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.393097:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05015 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.393106:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05015 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.393114:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760730.393119:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05015 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.393129:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05015 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.393148:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05015 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760730.393150:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972948 is committed 00010000:00010000:1.0:1789760730.393160:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05015 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.393246:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493702016:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39488us (39830us total) trans 4294972948 rc 0/0 00000100:00100000:1.0:1789760730.393257:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5659 00000100:00100000:1.0:1789760730.393267:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740652146218 time 740694778389 state 0x2 ref 0 00000100:00100000:2.0:1789760730.396514:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760730.396577:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493702144 00000100:00100000:1.0:1789760730.396588:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493702144 00000100:00100000:1.0:1789760730.396661:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740652146218 time 740694778389 state 0x0 ref 1 00000100:00100000:1.0:1789760730.396676:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740695038805 time 740694778389 state 0x1 ref 1 00000100:00100000:1.0:1789760730.396712:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740695038805 time 740737748182 state 0x0 ref 1 00000100:00100000:1.0:1789760730.396723:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493702144:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760730.396769:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23060480, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:1.0:1789760730.396777:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0501c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.396792:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760730.396796:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.396801:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0501c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.396814:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0501c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760730.430201:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972949 is committed 00002000:00010000:1.0:1789760730.430307:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23060480 -> 23064576 00010000:00010000:1.0:1789760730.430322:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0501c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.430339:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0501c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.430348:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0501c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.430356:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760730.430361:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0501c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.430370:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0501c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.430389:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0501c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.430395:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0501c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.430519:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493702144:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33792us (34009us total) trans 4294972949 rc 0/0 00000100:00100000:1.0:1789760730.430530:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5660 00000100:00100000:1.0:1789760730.430539:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740695038805 time 740737748182 state 0x2 ref 0 00000100:00100000:3.0:1789760730.435314:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760730.435383:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493702272 00000100:00100000:1.0:1789760730.435393:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493702272 00000100:00100000:1.0:1789760730.435452:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740695038805 time 740737748182 state 0x0 ref 1 00000100:00100000:1.0:1789760730.435467:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740738008598 time 740737748182 state 0x1 ref 1 00000100:00100000:1.0:1789760730.435499:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740738008598 time 740776534424 state 0x0 ref 1 00000100:00100000:1.0:1789760730.435512:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493702272:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760730.435571:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23064576, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:1.0:1789760730.435580:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05023 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.435596:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760730.435601:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.435607:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05023 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.435622:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05023 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760730.476257:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972950 is committed 00002000:00010000:1.0:1789760730.476434:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23064576 -> 23068672 00010000:00010000:1.0:1789760730.476450:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05023 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.476463:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05023 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.476478:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05023 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.476486:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760730.476491:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05023 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.476501:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05023 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.476520:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05023 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.476528:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05023 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.476697:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493702272:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41182us (41387us total) trans 4294972950 rc 0/0 00000100:00100000:1.0:1789760730.476708:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5661 00000100:00100000:1.0:1789760730.476717:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740738008598 time 740776534424 state 0x2 ref 0 00000100:00100000:0.0:1789760730.479608:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760730.479742:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493702400 00000100:00100000:2.0:1789760730.479750:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493702400 00000100:00100000:2.0:1789760730.479802:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740738008598 time 740776534424 state 0x0 ref 1 00000100:00100000:2.0:1789760730.479815:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740776794840 time 740776534424 state 0x1 ref 1 00000100:00100000:2.0:1789760730.479876:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740776794840 time 740820911135 state 0x0 ref 1 00000100:00100000:2.0:1789760730.479889:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493702400:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760730.479935:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23068672, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb007 00010000:00010000:2.0:1789760730.479944:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0502a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.479986:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760730.479992:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.479999:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0502a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.480013:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0502a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760730.519026:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23068672 -> 23072768 00002000:00010000:3.0:1789760730.519035:0:6692:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb blocks from disk: 45063 -> 47111 00010000:00010000:3.0:1789760730.519056:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0502a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.519068:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0502a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.519077:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0502a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.519090:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760730.519095:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0502a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.519103:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0502a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.519121:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0502a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.519129:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0502a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.519220:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493702400:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39328us (39616us total) trans 4294972951 rc 0/0 00000100:00100000:3.0:1789760730.519230:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5662 00000100:00100000:3.0:1789760730.519240:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740776794840 time 740820911135 state 0x2 ref 0 00000001:00080000:0.0:1789760730.520545:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972951 is committed 00000100:00100000:2.0:1789760730.523032:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760730.523310:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493702528 00000100:00100000:0.0:1789760730.523319:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493702528 00000100:00100000:0.0:1789760730.523370:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740776794840 time 740820911135 state 0x0 ref 1 00000100:00100000:0.0:1789760730.523384:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740821171551 time 740820911135 state 0x1 ref 1 00000100:00100000:0.0:1789760730.523413:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740821171551 time 740864448487 state 0x0 ref 1 00000100:00100000:0.0:1789760730.523424:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493702528:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760730.523470:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23072768, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:0.0:1789760730.523479:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05031 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.523494:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760730.523499:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.523505:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05031 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.523517:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05031 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760730.549748:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972952 is committed 00002000:00010000:1.0:1789760730.549916:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23072768 -> 23076864 00010000:00010000:1.0:1789760730.549931:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05031 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.549947:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05031 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.549956:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05031 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.549984:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760730.549988:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05031 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.549997:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05031 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.550017:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05031 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.550024:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05031 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.550176:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493702528:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26748us (27147us total) trans 4294972952 rc 0/0 00000100:00100000:1.0:1789760730.550187:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5663 00000100:00100000:1.0:1789760730.550197:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740821171551 time 740864448487 state 0x2 ref 0 00000100:00100000:2.0:1789760730.553510:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760730.553881:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493702656 00000100:00100000:1.0:1789760730.553892:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493702656 00000100:00100000:1.0:1789760730.553946:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740821171551 time 740864448487 state 0x0 ref 1 00000100:00100000:1.0:1789760730.553977:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740864708903 time 740864448487 state 0x1 ref 1 00000100:00100000:1.0:1789760730.554416:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740864708903 time 740895449626 state 0x0 ref 1 00000100:00100000:1.0:1789760730.554433:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493702656:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760730.554483:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23076864, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:1.0:1789760730.554491:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05038 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.554506:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760730.554511:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.554517:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05038 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.554531:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05038 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760730.581111:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972953 is committed 00002000:00010000:3.0:1789760730.581301:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23076864 -> 23080960 00010000:00010000:3.0:1789760730.581316:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05038 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.581329:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05038 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.581344:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05038 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.581352:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760730.581358:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05038 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.581367:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05038 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.581385:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05038 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.581392:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05038 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.581521:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493702656:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27088us (28015us total) trans 4294972953 rc 0/0 00000100:00100000:3.0:1789760730.581531:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5664 00000100:00100000:3.0:1789760730.581541:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740864708903 time 740895449626 state 0x2 ref 0 00000100:00100000:0.0:1789760730.586281:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760730.586615:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493702784 00000100:00100000:1.0:1789760730.586689:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493702784 00000100:00100000:1.0:1789760730.586742:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740864708903 time 740895449626 state 0x0 ref 1 00000100:00100000:1.0:1789760730.586756:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740895710042 time 740895449626 state 0x1 ref 1 00000100:00100000:1.0:1789760730.586891:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740895710042 time 740927925615 state 0x0 ref 1 00000100:00100000:1.0:1789760730.586906:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493702784:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760730.586953:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23080960, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xb807 00010000:00010000:1.0:1789760730.586988:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0503f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.587003:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760730.587008:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.587013:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0503f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.587028:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0503f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760730.623886:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972954 is committed 00002000:00010000:1.0:1789760730.624588:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23080960 -> 23085056 00002000:00010000:1.0:1789760730.624593:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760729 -> 1789760730 00002000:00010000:1.0:1789760730.624596:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760729 -> 1789760730 00010000:00010000:1.0:1789760730.624609:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0503f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.624674:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0503f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.624684:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0503f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.624691:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760730.624697:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0503f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.624705:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0503f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.624725:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0503f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.624731:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0503f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.624866:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493702784:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37959us (38590us total) trans 4294972954 rc 0/0 00000100:00100000:1.0:1789760730.624877:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5665 00000100:00100000:1.0:1789760730.625029:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740895710042 time 740927925615 state 0x2 ref 0 00000100:00100000:2.0:1789760730.629297:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760730.629404:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493702912 00000100:00100000:3.0:1789760730.629413:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493702912 00000100:00100000:3.0:1789760730.629538:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740895710042 time 740927925615 state 0x0 ref 1 00000100:00100000:3.0:1789760730.629555:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740928186031 time 740927925615 state 0x1 ref 1 00000100:00100000:3.0:1789760730.629806:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740928186031 time 740970840192 state 0x0 ref 1 00000100:00100000:3.0:1789760730.629822:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493702912:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760730.629870:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23085056, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:3.0:1789760730.629878:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05046 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.629980:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760730.629985:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760730.629991:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05046 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.630005:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05046 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760730.668538:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972955 is committed 00002000:00010000:3.0:1789760730.668646:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23085056 -> 23089152 00010000:00010000:3.0:1789760730.668662:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05046 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.668675:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05046 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.668683:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05046 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.668694:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760730.668699:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05046 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.668708:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05046 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.668729:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05046 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.668736:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05046 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.668837:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493702912:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39014us (39543us total) trans 4294972955 rc 0/0 00000100:00100000:3.0:1789760730.668848:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5666 00000100:00100000:3.0:1789760730.668857:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740928186031 time 740970840192 state 0x2 ref 0 00000100:00100000:2.0:1789760730.674419:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760730.674685:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493703040 00000100:00100000:0.0:1789760730.674696:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493703040 00000100:00100000:0.0:1789760730.674754:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740928186031 time 740970840192 state 0x0 ref 1 00000100:00100000:0.0:1789760730.674774:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740971100608 time 740970840192 state 0x1 ref 1 00000100:00100000:0.0:1789760730.674845:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740971100608 time 741015879723 state 0x0 ref 1 00000100:00100000:0.0:1789760730.674858:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ead80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493703040:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760730.674906:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23089152, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:0.0:1789760730.674916:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0504d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.674930:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760730.674934:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.674941:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0504d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.674956:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0504d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.685986:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760730.701488:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23089152 -> 23093248 00010000:00010000:1.0:1789760730.701507:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0504d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.701520:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0504d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.701529:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0504d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.701537:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760730.701542:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0504d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.701550:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0504d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.701579:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0504d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.701586:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0504d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.701683:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ead80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493703040:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26822us (27267us total) trans 4294972956 rc 0/0 00000100:00100000:1.0:1789760730.701693:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5667 00000100:00100000:1.0:1789760730.701703:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740971100608 time 741015879723 state 0x2 ref 0 00000001:00080000:3.0:1789760730.706223:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972956 is committed 00000100:00100000:0.0:1789760730.712268:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760730.712365:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493703168 00000100:00100000:1.0:1789760730.712374:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493703168 00000100:00100000:1.0:1789760730.712429:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 740971100608 time 741015879723 state 0x0 ref 1 00000100:00100000:1.0:1789760730.712444:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741016140139 time 741015879723 state 0x1 ref 1 00000100:00100000:1.0:1789760730.712484:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741016140139 time 741053519454 state 0x0 ref 1 00000100:00100000:1.0:1789760730.712501:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493703168:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760730.712551:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23093248, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:1.0:1789760730.712570:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05054 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.712588:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760730.712593:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.712598:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05054 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.712613:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05054 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760730.741528:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23093248 -> 23097344 00010000:00010000:0.0:1789760730.741549:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05054 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.741571:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05054 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.741581:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05054 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.741591:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760730.741597:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05054 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.741605:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05054 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.741624:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05054 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.741630:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05054 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.741734:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493703168:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29221us (29461us total) trans 4294972957 rc 0/0 00000100:00100000:0.0:1789760730.741744:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5668 00000100:00100000:0.0:1789760730.741754:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741016140139 time 741053519454 state 0x2 ref 0 00000001:00080000:2.0:1789760730.741805:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972957 is committed 00000100:00100000:0.0:1789760730.746078:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760730.746574:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493703296 00000100:00100000:1.0:1789760730.746586:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493703296 00000100:00100000:1.0:1789760730.746644:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741016140139 time 741053519454 state 0x0 ref 1 00000100:00100000:1.0:1789760730.746659:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741053779870 time 741053519454 state 0x1 ref 1 00000100:00100000:1.0:1789760730.746690:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741053779870 time 741087726108 state 0x0 ref 1 00000100:00100000:1.0:1789760730.746703:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb06050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493703296:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760730.746763:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23097344, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:1.0:1789760730.746772:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0505b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.746791:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760730.746795:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.746801:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0505b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.746812:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0505b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760730.795957:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972958 is committed 00002000:00010000:0.0:1789760730.796336:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23097344 -> 23101440 00010000:00010000:0.0:1789760730.796352:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0505b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.796366:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0505b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.796380:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0505b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.796387:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760730.796392:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0505b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.796402:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0505b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.796431:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0505b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.796440:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0505b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.796576:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb06050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493703296:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49871us (50619us total) trans 4294972958 rc 0/0 00000100:00100000:0.0:1789760730.796587:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5669 00000100:00100000:0.0:1789760730.796596:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741053779870 time 741087726108 state 0x2 ref 0 00000100:00100000:1.0:1789760730.802926:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760730.803055:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493703424 00000100:00100000:2.0:1789760730.803065:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493703424 00000100:00100000:2.0:1789760730.803122:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741053779870 time 741087726108 state 0x0 ref 1 00000100:00100000:2.0:1789760730.803148:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741087986524 time 741087726108 state 0x1 ref 1 00000100:00100000:2.0:1789760730.803170:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741087986524 time 741144205640 state 0x0 ref 1 00000100:00100000:2.0:1789760730.803186:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493703424:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760730.803231:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23101440, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:2.0:1789760730.803240:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05062 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.803256:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760730.803262:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.803267:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05062 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.803279:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05062 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760730.845081:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23101440 -> 23105536 00010000:00010000:0.0:1789760730.845101:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05062 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.845120:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05062 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.845129:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05062 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.845137:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760730.845141:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05062 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.845150:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05062 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.845170:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05062 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.845178:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05062 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.845281:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493703424:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42088us (42351us total) trans 4294972959 rc 0/0 00000100:00100000:0.0:1789760730.845292:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5670 00000100:00100000:0.0:1789760730.845302:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741087986524 time 741144205640 state 0x2 ref 0 00000001:00080000:1.0:1789760730.846243:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972959 is committed 00000100:00100000:3.0:1789760730.851918:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760730.852022:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493703552 00000100:00100000:0.0:1789760730.852032:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493703552 00000100:00100000:0.0:1789760730.852094:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741087986524 time 741144205640 state 0x0 ref 1 00000100:00100000:0.0:1789760730.852109:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741144466056 time 741144205640 state 0x1 ref 1 00000100:00100000:0.0:1789760730.852141:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741144466056 time 741193176884 state 0x0 ref 1 00000100:00100000:0.0:1789760730.852154:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493703552:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760730.852202:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23105536, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:0.0:1789760730.852210:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05069 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.852226:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760730.852231:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.852236:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05069 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.852249:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05069 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760730.881147:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972960 is committed 00002000:00010000:0.0:1789760730.881270:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23105536 -> 23109632 00010000:00010000:0.0:1789760730.881286:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05069 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.881298:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05069 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.881313:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05069 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.881321:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760730.881326:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05069 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.881336:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05069 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.881355:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05069 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.881364:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05069 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.881451:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493703552:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29294us (29538us total) trans 4294972960 rc 0/0 00000100:00100000:0.0:1789760730.881461:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5671 00000100:00100000:0.0:1789760730.881471:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741144466056 time 741193176884 state 0x2 ref 0 00000100:00100000:2.0:1789760730.888152:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760730.888246:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493703680 00000100:00100000:0.0:1789760730.888257:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493703680 00000100:00100000:0.0:1789760730.888311:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741144466056 time 741193176884 state 0x0 ref 1 00000100:00100000:0.0:1789760730.888326:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741193437300 time 741193176884 state 0x1 ref 1 00000100:00100000:0.0:1789760730.888390:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741193437300 time 741229425369 state 0x0 ref 1 00000100:00100000:0.0:1789760730.888402:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493703680:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760730.888452:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23109632, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:0.0:1789760730.888461:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05070 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.888476:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760730.888480:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.888486:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05070 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.888499:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05070 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760730.922751:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972961 is committed 00002000:00010000:2.0:1789760730.923280:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23109632 -> 23113728 00010000:00010000:2.0:1789760730.923296:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05070 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.923309:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05070 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.923318:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05070 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.923325:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760730.923329:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05070 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.923338:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05070 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.923357:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05070 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.923365:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05070 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.923504:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493703680:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35097us (35356us total) trans 4294972961 rc 0/0 00000100:00100000:2.0:1789760730.923514:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5672 00000100:00100000:2.0:1789760730.923525:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741193437300 time 741229425369 state 0x2 ref 0 00000100:00100000:1.0:1789760730.927463:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760730.927831:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493703808 00000100:00100000:0.0:1789760730.927841:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493703808 00000100:00100000:0.0:1789760730.927893:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741193437300 time 741229425369 state 0x0 ref 1 00000100:00100000:0.0:1789760730.927908:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741229685785 time 741229425369 state 0x1 ref 1 00000100:00100000:1.0:1789760730.928036:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741229685785 time 741269069036 state 0x0 ref 1 00000100:00100000:1.0:1789760730.928052:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493703808:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760730.928354:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23113728, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:1.0:1789760730.928365:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05077 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.928380:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760730.928385:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.928391:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05077 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.928405:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05077 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760730.967890:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972962 is committed 00002000:00010000:2.0:1789760730.968185:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23113728 -> 23117824 00010000:00010000:2.0:1789760730.968202:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05077 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.968215:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05077 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.968223:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05077 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.968230:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760730.968234:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05077 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.968244:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05077 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.968268:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05077 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.968276:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05077 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.968367:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493703808:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40315us (40909us total) trans 4294972962 rc 0/0 00000100:00100000:2.0:1789760730.968378:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5673 00000100:00100000:2.0:1789760730.968388:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741229685785 time 741269069036 state 0x2 ref 0 00000100:00100000:3.0:1789760730.973325:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760730.973445:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493703936 00000100:00100000:2.0:1789760730.973455:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493703936 00000100:00100000:2.0:1789760730.973512:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741229685785 time 741269069036 state 0x0 ref 1 00000100:00100000:2.0:1789760730.973528:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741269329452 time 741269069036 state 0x1 ref 1 00000100:00100000:2.0:1789760730.973568:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741269329452 time 741314603556 state 0x0 ref 1 00000100:00100000:2.0:1789760730.973581:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493703936:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760730.973627:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23117824, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:2.0:1789760730.973635:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0507e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.973650:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760730.973655:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.973661:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0507e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.973674:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0507e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760731.018778:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972963 is committed 00002000:00010000:1.0:1789760731.018893:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23117824 -> 23121920 00010000:00010000:1.0:1789760731.018910:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0507e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.018923:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0507e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.018933:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0507e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.018941:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760731.018945:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0507e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.018955:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0507e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.019005:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0507e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.019013:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0507e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.019096:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493703936:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45513us (45777us total) trans 4294972963 rc 0/0 00000100:00100000:1.0:1789760731.019108:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5674 00000100:00100000:1.0:1789760731.019117:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741269329452 time 741314603556 state 0x2 ref 0 00000100:00100000:1.0:1789760731.026583:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760731.030020:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493704064 00000100:00100000:0.0:1789760731.030035:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493704064 00000100:00100000:0.0:1789760731.030180:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741269329452 time 741314603556 state 0x0 ref 1 00000100:00100000:0.0:1789760731.030195:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741314863972 time 741314603556 state 0x1 ref 1 00000100:00100000:0.0:1789760731.030231:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741314863972 time 741371266154 state 0x0 ref 1 00000100:00100000:0.0:1789760731.030245:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493704064:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760731.030297:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23121920, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:0.0:1789760731.030306:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05085 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.030321:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760731.030326:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760731.030332:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05085 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.030356:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05085 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760731.035486:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760731.084463:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972964 is committed 00002000:00010000:2.0:1789760731.090414:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23121920 -> 23126016 00010000:00010000:2.0:1789760731.090435:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05085 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.090449:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05085 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.090457:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05085 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.090465:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760731.090469:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05085 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.090478:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05085 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.090498:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05085 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.090511:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05085 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.090720:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493704064:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60474us (64142us total) trans 4294972964 rc 0/0 00000100:00100000:2.0:1789760731.090730:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5675 00000100:00100000:2.0:1789760731.090740:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741314863972 time 741371266154 state 0x2 ref 0 00000100:00100000:3.0:1789760731.100520:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760731.100785:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493704192 00000100:00100000:2.0:1789760731.100795:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493704192 00000100:00100000:2.0:1789760731.100851:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741314863972 time 741371266154 state 0x0 ref 1 00000100:00100000:2.0:1789760731.100866:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741371526570 time 741371266154 state 0x1 ref 1 00000100:00100000:2.0:1789760731.100953:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741371526570 time 741441987684 state 0x0 ref 1 00000100:00100000:2.0:1789760731.100998:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493704192:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760731.101049:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23126016, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:2.0:1789760731.101058:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0508c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.101074:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760731.101079:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.101086:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0508c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.101104:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0508c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760731.137012:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23126016 -> 23130112 00010000:00010000:0.0:1789760731.137033:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0508c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.137047:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0508c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.137056:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0508c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.137065:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760731.137071:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0508c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.137080:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0508c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.137100:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0508c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.137109:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0508c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760731.137193:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493704192:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36194us (36679us total) trans 4294972965 rc 0/0 00000100:00100000:0.0:1789760731.137204:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5676 00000100:00100000:0.0:1789760731.137214:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741371526570 time 741441987684 state 0x2 ref 0 00000001:00080000:1.0:1789760731.137461:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972965 is committed 00000100:00100000:3.0:1789760731.149023:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760731.150670:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493704320 00000100:00100000:1.0:1789760731.150683:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493704320 00000100:00100000:1.0:1789760731.150749:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741371526570 time 741441987684 state 0x0 ref 1 00000100:00100000:1.0:1789760731.150765:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741442248100 time 741441987684 state 0x1 ref 1 00000100:00100000:1.0:1789760731.150800:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741442248100 time 741491836242 state 0x0 ref 1 00000100:00100000:1.0:1789760731.150813:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493704320:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760731.150864:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23130112, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:1.0:1789760731.150873:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05093 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.150889:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760731.150894:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.150901:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05093 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.150916:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05093 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760731.193305:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972966 is committed 00002000:00010000:1.0:1789760731.194080:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23130112 -> 23134208 00010000:00010000:1.0:1789760731.194099:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05093 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.194112:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05093 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.194122:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05093 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.194129:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760731.194134:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05093 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.194143:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05093 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.194162:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05093 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.194171:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05093 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.194272:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493704320:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43457us (45252us total) trans 4294972966 rc 0/0 00000100:00100000:1.0:1789760731.194288:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5677 00000100:00100000:1.0:1789760731.194299:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741442248100 time 741491836242 state 0x2 ref 0 00000100:00100000:2.0:1789760731.197539:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760731.197708:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493704448 00000100:00100000:1.0:1789760731.197717:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493704448 00000100:00100000:1.0:1789760731.197774:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741442248100 time 741491836242 state 0x0 ref 1 00000100:00100000:1.0:1789760731.197790:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741492096658 time 741491836242 state 0x1 ref 1 00000100:00100000:1.0:1789760731.197826:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741492096658 time 741538860916 state 0x0 ref 1 00000100:00100000:1.0:1789760731.197840:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493704448:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760731.197891:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23134208, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:1.0:1789760731.197900:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0509a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.197917:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760731.197923:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.197930:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0509a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.197946:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0509a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760731.258183:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972967 is committed 00002000:00010000:0.0:1789760731.258287:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23134208 -> 23138304 00010000:00010000:0.0:1789760731.258304:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0509a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.258318:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0509a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.258328:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0509a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.258353:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760731.258359:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0509a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.258369:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0509a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.258389:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0509a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.258397:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0509a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760731.258497:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493704448:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60655us (60962us total) trans 4294972967 rc 0/0 00000100:00100000:0.0:1789760731.258509:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5678 00000100:00100000:0.0:1789760731.258520:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741492096658 time 741538860916 state 0x2 ref 0 00000100:00100000:3.0:1789760731.262022:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760731.262133:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493704576 00000100:00100000:2.0:1789760731.262143:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493704576 00000100:00100000:2.0:1789760731.262268:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741492096658 time 741538860916 state 0x0 ref 1 00000100:00100000:2.0:1789760731.262287:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741539121332 time 741538860916 state 0x1 ref 1 00000100:00100000:2.0:1789760731.262381:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741539121332 time 741603414771 state 0x0 ref 1 00000100:00100000:2.0:1789760731.262395:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493704576:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760731.262446:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23138304, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:2.0:1789760731.262455:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e050a1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.262469:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760731.262474:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.262480:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e050a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.262494:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e050a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760731.310782:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23138304 -> 23142400 00010000:00010000:3.0:1789760731.310873:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e050a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.310887:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e050a1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.310896:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e050a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.310904:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760731.310909:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e050a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.310919:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e050a1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.310941:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e050a1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.310948:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e050a1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.311117:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493704576:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48719us (49101us total) trans 4294972968 rc 0/0 00000100:00100000:3.0:1789760731.311133:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5679 00000100:00100000:3.0:1789760731.311143:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741539121332 time 741603414771 state 0x2 ref 0 00000001:00080000:1.0:1789760731.311418:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972968 is committed 00000100:00100000:3.0:1789760731.315584:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760731.315758:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493704704 00000100:00100000:0.0:1789760731.315768:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493704704 00000100:00100000:0.0:1789760731.315947:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741539121332 time 741603414771 state 0x0 ref 1 00000100:00100000:0.0:1789760731.317779:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741603675187 time 741603414771 state 0x1 ref 1 00000100:00100000:0.0:1789760731.318041:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741603675187 time 741659074562 state 0x0 ref 1 00000100:00100000:0.0:1789760731.318068:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493704704:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760731.318125:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23142400, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:0.0:1789760731.318134:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e050a8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.318150:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760731.318154:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760731.318160:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e050a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.318174:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e050a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760731.376410:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972969 is committed 00002000:00010000:0.0:1789760731.376498:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23142400 -> 23146496 00010000:00010000:0.0:1789760731.376513:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e050a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.376821:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e050a8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.376831:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e050a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.376839:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760731.376844:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e050a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.376853:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e050a8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.376875:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e050a8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.376883:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e050a8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760731.377067:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493704704:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59004us (61487us total) trans 4294972969 rc 0/0 00000100:00100000:0.0:1789760731.377077:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5680 00000100:00100000:0.0:1789760731.377087:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741603675187 time 741659074562 state 0x2 ref 0 00000100:00100000:0.0:1789760731.381830:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760731.381927:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493704832 00000100:00100000:1.0:1789760731.381938:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493704832 00000100:00100000:1.0:1789760731.382181:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741603675187 time 741659074562 state 0x0 ref 1 00000100:00100000:1.0:1789760731.382196:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741659334978 time 741659074562 state 0x1 ref 1 00000100:00100000:1.0:1789760731.382264:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741659334978 time 741723299076 state 0x0 ref 1 00000100:00100000:1.0:1789760731.382279:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493704832:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760731.382335:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23146496, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:1.0:1789760731.382345:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050af lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.382361:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760731.382366:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.382373:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.382388:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760731.448869:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972970 is committed 00002000:00010000:1.0:1789760731.449127:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23146496 -> 23150592 00010000:00010000:1.0:1789760731.449142:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.449155:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050af lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.449170:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.449177:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760731.449182:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.449191:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050af lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.449210:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050af lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.449218:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050af lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.449318:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493704832:12345-192.168.203.60@tcp:4:dd.0 Request processed in 67036us (67493us total) trans 4294972970 rc 0/0 00000100:00100000:1.0:1789760731.449329:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5681 00000100:00100000:1.0:1789760731.449341:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741659334978 time 741723299076 state 0x2 ref 0 00000100:00100000:2.0:1789760731.453416:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760731.453498:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493704960 00000100:00100000:1.0:1789760731.453508:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493704960 00000100:00100000:1.0:1789760731.453583:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741659334978 time 741723299076 state 0x0 ref 1 00000100:00100000:1.0:1789760731.453598:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741723559492 time 741723299076 state 0x1 ref 1 00000100:00100000:1.0:1789760731.453631:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741723559492 time 741794666897 state 0x0 ref 1 00000100:00100000:1.0:1789760731.453645:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493704960:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760731.453693:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23150592, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:1.0:1789760731.453703:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e050b6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.453719:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760731.453723:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.453729:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e050b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.453744:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e050b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760731.494347:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23150592 -> 23154688 00010000:00010000:1.0:1789760731.494368:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e050b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.494380:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e050b6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.494388:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e050b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.494395:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760731.494400:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e050b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.494409:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e050b6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.494429:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e050b6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.494436:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e050b6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.494533:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493704960:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40885us (41122us total) trans 4294972971 rc 0/0 00000100:00100000:1.0:1789760731.494544:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5682 00000100:00100000:1.0:1789760731.494563:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741723559492 time 741794666897 state 0x2 ref 0 00000001:00080000:3.0:1789760731.495774:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972971 is committed 00000100:00100000:3.0:1789760731.498806:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760731.499233:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493705088 00000100:00100000:2.0:1789760731.499243:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493705088 00000100:00100000:2.0:1789760731.499301:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741723559492 time 741794666897 state 0x0 ref 1 00000100:00100000:2.0:1789760731.499318:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741794927313 time 741794666897 state 0x1 ref 1 00000100:00100000:1.0:1789760731.499392:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741794927313 time 741840420724 state 0x0 ref 1 00000100:00100000:1.0:1789760731.499409:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493705088:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760731.499463:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23154688, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:1.0:1789760731.499471:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050bd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.499486:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760731.499490:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.499499:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.499512:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760731.570980:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972972 is committed 00002000:00010000:1.0:1789760731.571070:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23154688 -> 23158784 00010000:00010000:1.0:1789760731.571086:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.571099:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050bd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.571108:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.571116:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760731.571122:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.571141:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050bd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.571161:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050bd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.571169:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050bd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.571266:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493705088:12345-192.168.203.60@tcp:4:dd.0 Request processed in 71858us (72466us total) trans 4294972972 rc 0/0 00000100:00100000:1.0:1789760731.571278:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5683 00000100:00100000:1.0:1789760731.571288:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741794927313 time 741840420724 state 0x2 ref 0 00000100:00100000:0.0:1789760731.578285:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760731.578389:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493705216 00000100:00100000:2.0:1789760731.578400:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493705216 00000100:00100000:2.0:1789760731.578468:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741794927313 time 741840420724 state 0x0 ref 1 00000100:00100000:2.0:1789760731.578483:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741840681140 time 741840420724 state 0x1 ref 1 00000100:00100000:2.0:1789760731.578507:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741840681140 time 741919542332 state 0x0 ref 1 00000100:00100000:2.0:1789760731.578532:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493705216:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760731.578593:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23158784, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xb807 00010000:00010000:2.0:1789760731.578603:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e050c4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.578628:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760731.578633:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.578638:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e050c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.578651:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e050c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760731.626683:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972973 is committed 00002000:00010000:3.0:1789760731.626943:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23158784 -> 23162880 00002000:00010000:3.0:1789760731.626948:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760730 -> 1789760731 00002000:00010000:3.0:1789760731.626950:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760730 -> 1789760731 00010000:00010000:3.0:1789760731.626979:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e050c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.626991:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e050c4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.626999:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e050c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.627006:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760731.627010:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e050c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.627019:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e050c4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.627042:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e050c4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.627050:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e050c4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.627312:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493705216:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48786us (49029us total) trans 4294972973 rc 0/0 00000100:00100000:3.0:1789760731.627333:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5684 00000100:00100000:3.0:1789760731.627343:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741840681140 time 741919542332 state 0x2 ref 0 00000100:00100000:2.0:1789760731.630255:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760731.630336:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493705344 00000100:00100000:3.0:1789760731.630346:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493705344 00000100:00100000:3.0:1789760731.630405:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741840681140 time 741919542332 state 0x0 ref 1 00000100:00100000:3.0:1789760731.630421:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741919802748 time 741919542332 state 0x1 ref 1 00000100:00100000:1.0:1789760731.630500:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741919802748 time 741971533144 state 0x0 ref 1 00000100:00100000:1.0:1789760731.630515:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493705344:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760731.630573:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23162880, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:1.0:1789760731.630582:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050cb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.630598:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760731.630602:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.630607:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.630621:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760731.668044:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972974 is committed 00002000:00010000:1.0:1789760731.668286:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23162880 -> 23166976 00010000:00010000:1.0:1789760731.668301:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.668320:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050cb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.668328:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.668336:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760731.668341:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.668351:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050cb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.668372:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050cb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.668380:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050cb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.668498:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493705344:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37981us (38247us total) trans 4294972974 rc 0/0 00000100:00100000:1.0:1789760731.668509:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5685 00000100:00100000:1.0:1789760731.668519:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741919802748 time 741971533144 state 0x2 ref 0 00000100:00100000:0.0:1789760731.676632:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760731.676729:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493705472 00000100:00100000:1.0:1789760731.676739:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493705472 00000100:00100000:1.0:1789760731.676794:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741919802748 time 741971533144 state 0x0 ref 1 00000100:00100000:1.0:1789760731.676810:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741971793560 time 741971533144 state 0x1 ref 1 00000100:00100000:1.0:1789760731.676840:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741971793560 time 742017875993 state 0x0 ref 1 00000100:00100000:1.0:1789760731.676853:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493705472:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760731.676902:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23166976, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:1.0:1789760731.676910:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050d2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.676925:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760731.676929:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.676935:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.676948:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.706383:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760731.713563:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972975 is committed 00002000:00010000:2.0:1789760731.713657:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23166976 -> 23171072 00010000:00010000:2.0:1789760731.713672:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.713684:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050d2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.713692:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.713700:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760731.713704:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.713712:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050d2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.713731:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050d2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.713739:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e050d2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.713825:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493705472:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36971us (37198us total) trans 4294972975 rc 0/0 00000100:00100000:2.0:1789760731.713835:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5686 00000100:00100000:2.0:1789760731.713844:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741971793560 time 742017875993 state 0x2 ref 0 00000100:00100000:2.0:1789760731.721740:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760731.721832:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493705600 00000100:00100000:3.0:1789760731.721840:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493705600 00000100:00100000:3.0:1789760731.722200:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 741971793560 time 742017875993 state 0x0 ref 1 00000100:00100000:3.0:1789760731.722219:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742018136409 time 742017875993 state 0x1 ref 1 00000100:00100000:3.0:1789760731.722299:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742018136409 time 742063333859 state 0x0 ref 1 00000100:00100000:3.0:1789760731.722312:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493705600:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760731.722361:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23171072, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:3.0:1789760731.722371:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050d9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.722386:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760731.722390:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.722395:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.722409:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760731.790913:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972976 is committed 00002000:00010000:3.0:1789760731.791090:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23171072 -> 23175168 00010000:00010000:3.0:1789760731.791105:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.791118:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050d9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.791126:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.791134:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760731.791138:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.791148:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050d9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.791168:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050d9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.791176:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050d9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.791280:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493705600:12345-192.168.203.60@tcp:4:dd.0 Request processed in 68958us (69535us total) trans 4294972976 rc 0/0 00000100:00100000:3.0:1789760731.791291:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5687 00000100:00100000:3.0:1789760731.791302:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742018136409 time 742063333859 state 0x2 ref 0 00000100:00100000:1.0:1789760731.797009:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760731.797146:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493705728 00000100:00100000:3.0:1789760731.797156:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493705728 00000100:00100000:3.0:1789760731.797207:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742018136409 time 742063333859 state 0x0 ref 1 00000100:00100000:3.0:1789760731.797224:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742063594275 time 742063333859 state 0x1 ref 1 00000100:00100000:3.0:1789760731.797288:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742063594275 time 742138322935 state 0x0 ref 1 00000100:00100000:3.0:1789760731.797300:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493705728:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760731.797346:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23175168, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:3.0:1789760731.797354:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050e0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.797370:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760731.797376:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.797382:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.797396:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760731.846036:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972977 is committed 00002000:00010000:3.0:1789760731.846527:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23175168 -> 23179264 00010000:00010000:3.0:1789760731.846542:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.846564:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050e0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.846573:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.846581:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760731.846586:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.846594:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050e0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.846675:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050e0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.846685:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050e0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.846811:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493705728:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49507us (49804us total) trans 4294972977 rc 0/0 00000100:00100000:3.0:1789760731.846821:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5688 00000100:00100000:3.0:1789760731.846830:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742063594275 time 742138322935 state 0x2 ref 0 00000100:00100000:0.0:1789760731.850933:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760731.851057:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493705856 00000100:00100000:1.0:1789760731.851066:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493705856 00000100:00100000:1.0:1789760731.851130:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742063594275 time 742138322935 state 0x0 ref 1 00000100:00100000:1.0:1789760731.851145:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742138583351 time 742138322935 state 0x1 ref 1 00000100:00100000:1.0:1789760731.851169:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742138583351 time 742192205264 state 0x0 ref 1 00000100:00100000:1.0:1789760731.851181:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493705856:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760731.851238:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23179264, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:1.0:1789760731.851247:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e050e7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.851263:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760731.851267:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.851273:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e050e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.851287:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e050e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760731.888122:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23179264 -> 23183360 00010000:00010000:1.0:1789760731.888142:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e050e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.888155:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e050e7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.888164:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e050e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.888173:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760731.888178:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e050e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.888188:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e050e7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.888207:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e050e7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.888214:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e050e7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.888307:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493705856:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37123us (37379us total) trans 4294972978 rc 0/0 00000100:00100000:1.0:1789760731.888318:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5689 00000001:00080000:0.0:1789760731.888399:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972978 is committed 00000100:00100000:1.0:1789760731.888440:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742138583351 time 742192205264 state 0x2 ref 0 00000100:00100000:2.0:1789760731.894092:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760731.894180:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493705984 00000100:00100000:1.0:1789760731.894191:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493705984 00000100:00100000:1.0:1789760731.894247:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742138583351 time 742192205264 state 0x0 ref 1 00000100:00100000:1.0:1789760731.894262:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742192465680 time 742192205264 state 0x1 ref 1 00000100:00100000:1.0:1789760731.894292:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742192465680 time 742235326856 state 0x0 ref 1 00000100:00100000:1.0:1789760731.894304:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493705984:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760731.894358:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23183360, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:1.0:1789760731.894366:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e050ee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.894382:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760731.894387:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.894393:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e050ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.894407:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e050ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760731.944430:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972979 is committed 00002000:00010000:2.0:1789760731.944518:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23183360 -> 23187456 00010000:00010000:2.0:1789760731.944533:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e050ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.944547:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e050ee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.944565:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e050ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.944572:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760731.944577:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e050ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.944586:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e050ee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.944925:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e050ee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.945216:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e050ee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.945356:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493705984:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51049us (51268us total) trans 4294972979 rc 0/0 00000100:00100000:2.0:1789760731.945367:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5690 00000100:00100000:2.0:1789760731.945377:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742192465680 time 742235326856 state 0x2 ref 0 00000100:00100000:1.0:1789760731.950599:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760731.950781:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493706112 00000100:00100000:3.0:1789760731.950792:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493706112 00000100:00100000:3.0:1789760731.950845:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742192465680 time 742235326856 state 0x0 ref 1 00000100:00100000:3.0:1789760731.950859:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742235587272 time 742235326856 state 0x1 ref 1 00000100:00100000:3.0:1789760731.950887:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742235587272 time 742291923282 state 0x0 ref 1 00000100:00100000:3.0:1789760731.950900:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493706112:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760731.950948:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23187456, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:3.0:1789760731.950957:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050f5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.951002:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760731.951007:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.951013:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.951026:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760731.995745:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972980 is committed 00002000:00010000:3.0:1789760731.995860:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23187456 -> 23191552 00010000:00010000:3.0:1789760731.995875:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.995888:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050f5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.995897:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.995905:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760731.995910:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.995919:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050f5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.995938:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050f5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.995946:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e050f5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.996065:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493706112:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45161us (45470us total) trans 4294972980 rc 0/0 00000100:00100000:3.0:1789760731.996080:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5691 00000100:00100000:3.0:1789760731.996089:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742235587272 time 742291923282 state 0x2 ref 0 00000100:00100000:0.0:1789760732.000335:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760732.000457:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493706240 00000100:00100000:2.0:1789760732.000468:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493706240 00000100:00100000:2.0:1789760732.000522:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742235587272 time 742291923282 state 0x0 ref 1 00000100:00100000:2.0:1789760732.000537:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742292183698 time 742291923282 state 0x1 ref 1 00000100:00100000:2.0:1789760732.000577:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742292183698 time 742341612178 state 0x0 ref 1 00000100:00100000:2.0:1789760732.000589:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493706240:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760732.000644:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23191552, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:2.0:1789760732.000653:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e050fc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.000673:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760732.000677:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760732.000684:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e050fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.000699:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e050fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760732.040534:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972981 is committed 00002000:00010000:2.0:1789760732.040764:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23191552 -> 23195648 00010000:00010000:2.0:1789760732.040780:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e050fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.040794:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e050fc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.040803:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e050fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.040811:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760732.040815:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e050fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.040825:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e050fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.040845:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e050fc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.040854:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e050fc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760732.040940:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493706240:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40348us (40611us total) trans 4294972981 rc 0/0 00000100:00100000:2.0:1789760732.040951:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5692 00000100:00100000:2.0:1789760732.040995:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742292183698 time 742341612178 state 0x2 ref 0 00000100:00100000:2.0:1789760732.043468:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760732.043684:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493706368 00000100:00100000:1.0:1789760732.043694:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493706368 00000100:00100000:1.0:1789760732.043744:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742292183698 time 742341612178 state 0x0 ref 1 00000100:00100000:1.0:1789760732.043758:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742341872594 time 742341612178 state 0x1 ref 1 00000100:00100000:1.0:1789760732.043846:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742341872594 time 742384880476 state 0x0 ref 1 00000100:00100000:1.0:1789760732.043859:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493706368:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760732.043907:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23195648, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:1.0:1789760732.043916:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05103 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.043931:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760732.043936:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.043942:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05103 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.043957:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05103 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760732.073881:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972982 is committed 00002000:00010000:3.0:1789760732.076265:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23195648 -> 23199744 00010000:00010000:3.0:1789760732.076284:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05103 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.076298:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05103 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.076307:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05103 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.076314:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760732.076319:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05103 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.076330:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05103 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.076348:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05103 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.076355:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05103 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.076462:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493706368:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32601us (32999us total) trans 4294972982 rc 0/0 00000100:00100000:3.0:1789760732.076473:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5693 00000100:00100000:3.0:1789760732.076483:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742341872594 time 742384880476 state 0x2 ref 0 00000100:00100000:1.0:1789760732.083885:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760732.084169:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493706496 00000100:00100000:2.0:1789760732.084180:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493706496 00000100:00100000:2.0:1789760732.084238:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742341872594 time 742384880476 state 0x0 ref 1 00000100:00100000:2.0:1789760732.084253:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742385140892 time 742384880476 state 0x1 ref 1 00000100:00100000:2.0:1789760732.084324:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742385140892 time 742425359360 state 0x0 ref 1 00000100:00100000:2.0:1789760732.084337:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493706496:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760732.084389:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23199744, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:2.0:1789760732.084397:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0510a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.084414:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760732.084418:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760732.084426:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0510a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.084440:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0510a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760732.129875:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972983 is committed 00002000:00010000:3.0:1789760732.130014:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23199744 -> 23203840 00010000:00010000:3.0:1789760732.130029:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0510a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.130042:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0510a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.130052:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0510a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.130059:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760732.130064:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0510a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.130073:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0510a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.130108:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0510a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.130120:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0510a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.130201:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493706496:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45863us (46323us total) trans 4294972983 rc 0/0 00000100:00100000:3.0:1789760732.130211:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5694 00000100:00100000:3.0:1789760732.130221:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742385140892 time 742425359360 state 0x2 ref 0 00000100:00100000:1.0:1789760732.133562:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760732.133642:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493706624 00000100:00100000:3.0:1789760732.133653:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493706624 00000100:00100000:3.0:1789760732.133703:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742385140892 time 742425359360 state 0x0 ref 1 00000100:00100000:3.0:1789760732.133718:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742425619776 time 742425359360 state 0x1 ref 1 00000100:00100000:3.0:1789760732.133751:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742425619776 time 742474786898 state 0x0 ref 1 00000100:00100000:3.0:1789760732.133763:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493706624:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760732.133810:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23203840, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:3.0:1789760732.133818:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05111 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.133834:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760732.133838:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.133844:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05111 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.133863:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05111 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760732.161449:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972984 is committed 00002000:00010000:0.0:1789760732.161531:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23203840 -> 23207936 00010000:00010000:0.0:1789760732.161545:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05111 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.161571:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05111 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.161579:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05111 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.161586:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760732.161591:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05111 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.161609:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05111 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.161630:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05111 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.161637:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05111 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.161735:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493706624:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27968us (28189us total) trans 4294972984 rc 0/0 00000100:00100000:0.0:1789760732.161746:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5695 00000100:00100000:0.0:1789760732.161757:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742425619776 time 742474786898 state 0x2 ref 0 00000100:00100000:2.0:1789760732.166190:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760732.166270:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493706752 00000100:00100000:3.0:1789760732.166279:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493706752 00000100:00100000:3.0:1789760732.166332:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742425619776 time 742474786898 state 0x0 ref 1 00000100:00100000:3.0:1789760732.166345:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742475047314 time 742474786898 state 0x1 ref 1 00000100:00100000:3.0:1789760732.166379:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742475047314 time 742507414967 state 0x0 ref 1 00000100:00100000:3.0:1789760732.166391:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493706752:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760732.166441:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23207936, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:3.0:1789760732.166450:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05118 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.166467:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760732.166472:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.166478:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05118 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.166492:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05118 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760732.206091:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23207936 -> 23212032 00010000:00010000:3.0:1789760732.206114:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05118 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.206128:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05118 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.206138:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05118 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.206145:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760732.206150:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05118 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.206160:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05118 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.206179:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05118 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.206187:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05118 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.206282:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493706752:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39889us (40099us total) trans 4294972985 rc 0/0 00000100:00100000:3.0:1789760732.206292:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5696 00000100:00100000:3.0:1789760732.206302:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742475047314 time 742507414967 state 0x2 ref 0 00000001:00080000:0.0:1789760732.206956:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972985 is committed 00000100:00100000:2.0:1789760732.209818:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760732.209925:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493706880 00000100:00100000:1.0:1789760732.209936:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493706880 00000100:00100000:1.0:1789760732.210038:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742475047314 time 742507414967 state 0x0 ref 1 00000100:00100000:1.0:1789760732.210055:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742507675383 time 742507414967 state 0x1 ref 1 00000100:00100000:1.0:1789760732.210076:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742507675383 time 742551112022 state 0x0 ref 1 00000100:00100000:1.0:1789760732.210088:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493706880:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760732.210136:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23212032, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:1.0:1789760732.210145:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0511f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.210161:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760732.210166:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.210172:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0511f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.210185:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0511f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760732.245874:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972986 is committed 00002000:00010000:1.0:1789760732.246191:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23212032 -> 23216128 00010000:00010000:1.0:1789760732.246208:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0511f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.246221:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0511f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.246230:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0511f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.246237:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760732.246242:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0511f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.246251:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0511f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.246271:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0511f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.246279:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0511f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.246373:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493706880:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36281us (36558us total) trans 4294972986 rc 0/0 00000100:00100000:1.0:1789760732.246389:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5697 00000100:00100000:1.0:1789760732.246400:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742507675383 time 742551112022 state 0x2 ref 0 00000100:00100000:1.0:1789760732.252292:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760732.252378:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493707008 00000100:00100000:2.0:1789760732.252387:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493707008 00000100:00100000:2.0:1789760732.252446:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742507675383 time 742551112022 state 0x0 ref 1 00000100:00100000:2.0:1789760732.252461:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742551372438 time 742551112022 state 0x1 ref 1 00000100:00100000:2.0:1789760732.252492:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742551372438 time 742593528015 state 0x0 ref 1 00000100:00100000:2.0:1789760732.252634:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493707008:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760732.252685:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23216128, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:2.0:1789760732.252694:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05126 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.252710:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760732.252715:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760732.252721:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05126 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.252735:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05126 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.264146:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760732.264182:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760732.264247:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760732.264275:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493707264 00000100:00100000:3.0:1789760732.264283:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493707264 00000100:00100000:3.0:1789760732.264333:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493707392 00000100:00100000:3.0:1789760732.264336:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493707392 00000100:00100000:3.0:1789760732.264362:0:9460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 472 00000100:00100000:3.0:1789760732.264371:0:9460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2369:x1876699493707264:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:3.0:1789760732.264438:0:9460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2369:x1876699493707264:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 68us (302us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760732.264444:0:9460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 472 00000100:00100000:3.0:1789760732.264459:0:9460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 473 00000100:00100000:3.0:1789760732.264465:0:9460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2369:x1876699493707392:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:3.0:1789760732.264488:0:9460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2369:x1876699493707392:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 23us (240us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760732.264494:0:9460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 473 00000100:00100000:3.0:1789760732.264520:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493707136 00000100:00100000:3.0:1789760732.264524:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493707136 00000100:00100000:3.0:1789760732.264558:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 395 00000100:00100000:3.0:1789760732.264565:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d4ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2369:x1876699493707136:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:3.0:1789760732.264598:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d4ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2369:x1876699493707136:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 34us (415us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760732.264610:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 395 00000100:00080000:3.0:1789760732.264638:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:1.0:1789760732.290137:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972987 is committed 00002000:00010000:3.0:1789760732.290312:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23216128 -> 23220224 00010000:00010000:3.0:1789760732.290328:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05126 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.290342:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05126 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.290351:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05126 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.290359:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760732.290364:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05126 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.290374:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05126 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.290392:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05126 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.290400:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05126 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.290532:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493707008:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37898us (38244us total) trans 4294972987 rc 0/0 00000100:00100000:3.0:1789760732.290543:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5698 00000100:00100000:3.0:1789760732.290561:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742551372438 time 742593528015 state 0x2 ref 0 00000100:00100000:3.0:1789760732.295208:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760732.295676:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493707520 00000100:00100000:0.0:1789760732.295686:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493707520 00000100:00100000:0.0:1789760732.295738:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742551372438 time 742593528015 state 0x0 ref 1 00000100:00100000:0.0:1789760732.295753:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742593788431 time 742593528015 state 0x1 ref 1 00000100:00100000:0.0:1789760732.295839:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742593788431 time 742636873518 state 0x0 ref 1 00000100:00100000:0.0:1789760732.295851:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493707520:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760732.295897:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23220224, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:0.0:1789760732.295906:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0512d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.295919:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:0.0:1789760732.295924:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760732.295930:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0512d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.295943:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0512d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760732.329508:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23220224 -> 23224320 00010000:00010000:0.0:1789760732.329527:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0512d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.329542:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0512d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.329567:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0512d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.329583:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760732.329587:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0512d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.329595:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0512d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.329622:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0512d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.329630:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0512d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.329725:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493707520:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33870us (34522us total) trans 4294972988 rc 0/0 00000100:00100000:0.0:1789760732.329735:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5699 00000100:00100000:0.0:1789760732.329744:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742593788431 time 742636873518 state 0x2 ref 0 00000001:00080000:1.0:1789760732.330353:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972988 is committed 00000100:00100000:1.0:1789760732.333296:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760732.333378:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493707648 00000100:00100000:3.0:1789760732.333387:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493707648 00000100:00100000:3.0:1789760732.333440:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742593788431 time 742636873518 state 0x0 ref 1 00000100:00100000:3.0:1789760732.333455:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742637133934 time 742636873518 state 0x1 ref 1 00000100:00100000:3.0:1789760732.333531:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742637133934 time 742674566472 state 0x0 ref 1 00000100:00100000:3.0:1789760732.333543:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493707648:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760732.333612:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23224320, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:3.0:1789760732.333620:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05134 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.333635:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760732.333640:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.333646:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05134 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.333664:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05134 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760732.367739:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972989 is committed 00002000:00010000:3.0:1789760732.367780:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23224320 -> 23228416 00010000:00010000:3.0:1789760732.367797:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05134 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.367816:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05134 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.367825:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05134 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.367831:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760732.367836:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05134 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.367845:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05134 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.367864:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05134 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.367873:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05134 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.368010:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493707648:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34417us (34670us total) trans 4294972989 rc 0/0 00000100:00100000:3.0:1789760732.368020:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5700 00000100:00100000:3.0:1789760732.368030:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742637133934 time 742674566472 state 0x2 ref 0 00000100:00100000:3.0:1789760732.373203:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760732.373264:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493707776 00000100:00100000:1.0:1789760732.373275:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493707776 00000100:00100000:1.0:1789760732.373446:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742637133934 time 742674566472 state 0x0 ref 1 00000100:00100000:1.0:1789760732.373462:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742674826888 time 742674566472 state 0x1 ref 1 00000100:00100000:1.0:1789760732.373828:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742674826888 time 742714863066 state 0x0 ref 1 00000100:00100000:1.0:1789760732.373841:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493707776:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760732.373883:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23228416, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:1.0:1789760732.373890:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0513b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.373902:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760732.373905:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.373910:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0513b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.373924:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0513b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760732.415469:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972990 is committed 00002000:00010000:0.0:1789760732.415525:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23228416 -> 23232512 00010000:00010000:0.0:1789760732.415540:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0513b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.415569:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0513b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.415578:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0513b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.415585:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760732.415590:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0513b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.415713:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0513b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.415735:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0513b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.415743:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0513b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.415839:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493707776:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41996us (42641us total) trans 4294972990 rc 0/0 00000100:00100000:0.0:1789760732.415850:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5701 00000100:00100000:0.0:1789760732.415860:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742674826888 time 742714863066 state 0x2 ref 0 00000100:00100000:2.0:1789760732.422805:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760732.423349:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493707904 00000100:00100000:3.0:1789760732.423359:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493707904 00000100:00100000:3.0:1789760732.423418:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742674826888 time 742714863066 state 0x0 ref 1 00000100:00100000:3.0:1789760732.423435:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742715123482 time 742714863066 state 0x1 ref 1 00000100:00100000:3.0:1789760732.423467:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742715123482 time 742764502390 state 0x0 ref 1 00000100:00100000:3.0:1789760732.423480:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493707904:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760732.423530:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23232512, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:3.0:1789760732.423540:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05142 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.423568:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760732.423572:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.423579:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05142 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.423592:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05142 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760732.466186:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972991 is committed 00002000:00010000:3.0:1789760732.466561:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23232512 -> 23236608 00010000:00010000:3.0:1789760732.466671:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05142 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.466685:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05142 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.466699:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05142 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.466706:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760732.466711:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05142 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.466720:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05142 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.466741:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05142 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.466748:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05142 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.466823:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493707904:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43341us (44027us total) trans 4294972991 rc 0/0 00000100:00100000:3.0:1789760732.466832:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5702 00000100:00100000:3.0:1789760732.466842:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742715123482 time 742764502390 state 0x2 ref 0 00000100:00100000:1.0:1789760732.471092:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760732.471186:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493708032 00000100:00100000:0.0:1789760732.471196:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493708032 00000100:00100000:0.0:1789760732.471251:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742715123482 time 742764502390 state 0x0 ref 1 00000100:00100000:0.0:1789760732.471265:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742764762806 time 742764502390 state 0x1 ref 1 00000100:00100000:0.0:1789760732.471296:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742764762806 time 742812331609 state 0x0 ref 1 00000100:00100000:0.0:1789760732.471309:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493708032:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760732.471363:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23236608, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:0.0:1789760732.471383:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05149 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.471400:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:0.0:1789760732.471405:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760732.471412:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05149 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.471425:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05149 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760732.521523:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972992 is committed 00002000:00010000:0.0:1789760732.522044:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23236608 -> 23240704 00010000:00010000:0.0:1789760732.522061:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05149 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.522073:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05149 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.522087:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05149 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.522094:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760732.522099:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05149 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.522108:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05149 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.522128:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05149 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.522136:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05149 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.522233:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493708032:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50921us (51145us total) trans 4294972992 rc 0/0 00000100:00100000:0.0:1789760732.522244:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5703 00000100:00100000:0.0:1789760732.522253:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742764762806 time 742812331609 state 0x2 ref 0 00000100:00100000:2.0:1789760732.528068:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760732.528246:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493708160 00000100:00100000:3.0:1789760732.528257:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493708160 00000100:00100000:3.0:1789760732.528311:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742764762806 time 742812331609 state 0x0 ref 1 00000100:00100000:3.0:1789760732.528331:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742812592025 time 742812331609 state 0x1 ref 1 00000100:00100000:3.0:1789760732.528360:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742812592025 time 742869394846 state 0x0 ref 1 00000100:00100000:3.0:1789760732.528670:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493708160:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760732.528720:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23240704, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:3.0:1789760732.528730:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05150 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.528744:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760732.528748:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.528754:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05150 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.528769:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05150 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760732.574958:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23240704 -> 23244800 00010000:00010000:3.0:1789760732.574994:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05150 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.575008:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05150 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.575016:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05150 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.575030:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760732.575035:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05150 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.575043:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05150 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.575062:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05150 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.575071:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05150 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.575166:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493708160:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46791us (47199us total) trans 4294972993 rc 0/0 00000100:00100000:3.0:1789760732.575619:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5704 00000100:00100000:3.0:1789760732.575630:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742812592025 time 742869394846 state 0x2 ref 0 00000001:00080000:1.0:1789760732.575751:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972993 is committed 00000100:00100000:2.0:1789760732.578916:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760732.579130:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493708288 00000100:00100000:1.0:1789760732.579141:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493708288 00000100:00100000:2.0:1789760732.579304:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742812592025 time 742869394846 state 0x0 ref 1 00000100:00100000:2.0:1789760732.579320:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742869655262 time 742869394846 state 0x1 ref 1 00000100:00100000:2.0:1789760732.579424:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742869655262 time 742920458956 state 0x0 ref 1 00000100:00100000:2.0:1789760732.579438:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493708288:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760732.579487:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23244800, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xb807 00010000:00010000:2.0:1789760732.579496:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05157 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.579511:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760732.579516:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760732.579522:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05157 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.579537:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05157 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760732.608708:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972994 is committed 00002000:00010000:0.0:1789760732.608812:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23244800 -> 23248896 00002000:00010000:0.0:1789760732.608817:0:13038:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760731 -> 1789760732 00002000:00010000:0.0:1789760732.608820:0:13038:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760731 -> 1789760732 00010000:00010000:0.0:1789760732.608834:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05157 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.608846:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05157 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.608856:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05157 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.608865:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760732.608869:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05157 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.608878:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05157 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.608898:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05157 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.608904:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05157 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.609112:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493708288:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29672us (30200us total) trans 4294972994 rc 0/0 00000100:00100000:0.0:1789760732.609123:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5705 00000100:00100000:0.0:1789760732.609133:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742869655262 time 742920458956 state 0x2 ref 0 00000100:00100000:0.0:1789760732.617272:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760732.617383:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493708416 00000100:00100000:2.0:1789760732.617393:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493708416 00000100:00100000:2.0:1789760732.617451:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742869655262 time 742920458956 state 0x0 ref 1 00000100:00100000:2.0:1789760732.617467:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742920719372 time 742920458956 state 0x1 ref 1 00000100:00100000:2.0:1789760732.617497:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742920719372 time 742958532313 state 0x0 ref 1 00000100:00100000:2.0:1789760732.617510:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493708416:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760732.617576:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23248896, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:2.0:1789760732.617585:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0515e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.617600:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760732.617605:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760732.617611:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0515e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.617632:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0515e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760732.678446:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972995 is committed 00002000:00010000:2.0:1789760732.679084:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23248896 -> 23252992 00010000:00010000:2.0:1789760732.679099:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0515e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.679118:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0515e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.679128:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0515e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.679135:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760732.679140:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0515e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.679150:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0515e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.679171:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0515e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.679178:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0515e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760732.679311:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493708416:12345-192.168.203.60@tcp:4:dd.0 Request processed in 61798us (62139us total) trans 4294972995 rc 0/0 00000100:00100000:2.0:1789760732.679322:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5706 00000100:00100000:2.0:1789760732.679331:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742920719372 time 742958532313 state 0x2 ref 0 00000100:00100000:2.0:1789760732.683321:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760732.683426:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493708544 00000100:00100000:0.0:1789760732.683436:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493708544 00000100:00100000:0.0:1789760732.683493:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742920719372 time 742958532313 state 0x0 ref 1 00000100:00100000:0.0:1789760732.683508:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742958792729 time 742958532313 state 0x1 ref 1 00000100:00100000:0.0:1789760732.683575:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742958792729 time 743024610660 state 0x0 ref 1 00000100:00100000:0.0:1789760732.683588:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493708544:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760732.683651:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23252992, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:0.0:1789760732.683660:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05165 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.683675:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760732.683679:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760732.683686:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05165 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.683699:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05165 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760732.718453:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972996 is committed 00002000:00010000:0.0:1789760732.719297:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23252992 -> 23257088 00010000:00010000:0.0:1789760732.719315:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05165 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.719328:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05165 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.719344:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05165 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.719352:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760732.719356:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05165 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.719366:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05165 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.719388:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05165 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.719396:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05165 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.719498:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493708544:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35906us (36181us total) trans 4294972996 rc 0/0 00000100:00100000:0.0:1789760732.719508:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5707 00000100:00100000:0.0:1789760732.719518:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742958792729 time 743024610660 state 0x2 ref 0 00000100:00100000:0.0:1789760732.725013:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760732.725161:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493708672 00000100:00100000:3.0:1789760732.725171:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493708672 00000100:00100000:3.0:1789760732.725225:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 742958792729 time 743024610660 state 0x0 ref 1 00000100:00100000:3.0:1789760732.725240:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743024871076 time 743024610660 state 0x1 ref 1 00000100:00100000:3.0:1789760732.725323:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743024871076 time 743066357288 state 0x0 ref 1 00000100:00100000:3.0:1789760732.725336:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493708672:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760732.725383:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23257088, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:3.0:1789760732.725393:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0516c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.725409:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760732.725414:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.725420:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0516c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.725435:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0516c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.731591:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760732.751104:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972997 is committed 00002000:00010000:0.0:1789760732.751289:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23257088 -> 23261184 00010000:00010000:0.0:1789760732.751305:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0516c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.751320:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0516c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.751329:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0516c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.751337:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760732.751343:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0516c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.751354:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0516c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.751375:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0516c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.751383:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0516c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.751477:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493708672:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26137us (26468us total) trans 4294972997 rc 0/0 00000100:00100000:0.0:1789760732.751488:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5708 00000100:00100000:0.0:1789760732.751498:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743024871076 time 743066357288 state 0x2 ref 0 00000100:00100000:0.0:1789760732.754905:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760732.755178:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493708800 00000100:00100000:2.0:1789760732.755187:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493708800 00000100:00100000:2.0:1789760732.755242:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743024871076 time 743066357288 state 0x0 ref 1 00000100:00100000:2.0:1789760732.755257:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743066617704 time 743066357288 state 0x1 ref 1 00000100:00100000:2.0:1789760732.755293:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743066617704 time 743096328642 state 0x0 ref 1 00000100:00100000:2.0:1789760732.755305:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493708800:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760732.755352:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23261184, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:2.0:1789760732.755361:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05173 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.755376:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760732.755381:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760732.755387:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05173 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.755402:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05173 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760732.790263:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23261184 -> 23265280 00010000:00010000:2.0:1789760732.790286:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05173 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.790300:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05173 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.790312:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05173 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.790320:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760732.790325:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05173 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.790335:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05173 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.790354:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05173 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.790363:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05173 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760732.790446:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493708800:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35138us (35546us total) trans 4294972998 rc 0/0 00000100:00100000:2.0:1789760732.790457:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5709 00000100:00100000:2.0:1789760732.790467:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743066617704 time 743096328642 state 0x2 ref 0 00000001:00080000:2.0:1789760732.790505:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972998 is committed 00000100:00100000:2.0:1789760732.793999:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760732.794137:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493708928 00000100:00100000:3.0:1789760732.794148:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493708928 00000100:00100000:3.0:1789760732.794204:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743066617704 time 743096328642 state 0x0 ref 1 00000100:00100000:3.0:1789760732.794220:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743096589058 time 743096328642 state 0x1 ref 1 00000100:00100000:3.0:1789760732.794240:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743096589058 time 743135276273 state 0x0 ref 1 00000100:00100000:3.0:1789760732.794252:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493708928:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760732.794301:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23265280, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:3.0:1789760732.794311:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0517a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.794327:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760732.794330:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.794336:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0517a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.794349:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0517a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760732.844255:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23265280 -> 23269376 00010000:00010000:0.0:1789760732.844273:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0517a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.844285:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0517a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.844294:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0517a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.844301:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760732.844306:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0517a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.844315:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0517a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.844334:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0517a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.844341:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0517a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.844423:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493708928:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50168us (50457us total) trans 4294972999 rc 0/0 00000100:00100000:0.0:1789760732.844433:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5710 00000100:00100000:0.0:1789760732.844442:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743096589058 time 743135276273 state 0x2 ref 0 00000001:00080000:1.0:1789760732.844510:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972999 is committed 00000100:00100000:0.0:1789760732.847789:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760732.847870:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493709056 00000100:00100000:2.0:1789760732.847879:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493709056 00000100:00100000:2.0:1789760732.847948:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743096589058 time 743135276273 state 0x0 ref 1 00000100:00100000:2.0:1789760732.847990:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743135536689 time 743135276273 state 0x1 ref 1 00000100:00100000:2.0:1789760732.848019:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743135536689 time 743189054539 state 0x0 ref 1 00000100:00100000:2.0:1789760732.848041:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493709056:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760732.848090:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23269376, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:2.0:1789760732.848099:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05181 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.848114:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760732.848119:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760732.848125:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05181 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.848139:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05181 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760732.885924:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23269376 -> 23273472 00010000:00010000:0.0:1789760732.885957:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05181 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.885984:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05181 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.885994:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05181 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.886002:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760732.886008:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05181 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.886017:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05181 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.886045:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05181 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.886053:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05181 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.886143:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493709056:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38099us (38363us total) trans 4294973000 rc 0/0 00000100:00100000:0.0:1789760732.886157:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5711 00000100:00100000:0.0:1789760732.886167:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743135536689 time 743189054539 state 0x2 ref 0 00000001:00080000:3.0:1789760732.886580:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973000 is committed 00000100:00100000:3.0:1789760732.890534:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760732.890951:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493709184 00000100:00100000:0.0:1789760732.890996:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493709184 00000100:00100000:0.0:1789760732.891066:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743135536689 time 743189054539 state 0x0 ref 1 00000100:00100000:0.0:1789760732.891082:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743189314955 time 743189054539 state 0x1 ref 1 00000100:00100000:0.0:1789760732.891114:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743189314955 time 743232149594 state 0x0 ref 1 00000100:00100000:0.0:1789760732.891127:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493709184:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760732.891178:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23273472, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:0.0:1789760732.891188:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05188 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.891203:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760732.891207:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760732.891214:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05188 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.891227:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05188 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760732.926280:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973001 is committed 00002000:00010000:0.0:1789760732.926358:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23273472 -> 23277568 00010000:00010000:0.0:1789760732.926374:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05188 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.926387:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05188 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.926396:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05188 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.926403:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760732.926408:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05188 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.926417:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05188 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.926437:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05188 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.926445:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05188 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.926584:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493709184:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35453us (36054us total) trans 4294973001 rc 0/0 00000100:00100000:0.0:1789760732.926605:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5712 00000100:00100000:0.0:1789760732.926615:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743189314955 time 743232149594 state 0x2 ref 0 00000100:00100000:1.0:1789760732.930522:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760732.930601:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493709312 00000100:00100000:2.0:1789760732.930610:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493709312 00000100:00100000:2.0:1789760732.930661:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743189314955 time 743232149594 state 0x0 ref 1 00000100:00100000:2.0:1789760732.930675:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743232410010 time 743232149594 state 0x1 ref 1 00000100:00100000:2.0:1789760732.930704:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743232410010 time 743271740682 state 0x0 ref 1 00000100:00100000:2.0:1789760732.930716:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493709312:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760732.930770:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23277568, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:2.0:1789760732.930778:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0518f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.930794:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760732.930798:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760732.930803:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0518f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.930816:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0518f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760732.961527:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973002 is committed 00002000:00010000:2.0:1789760732.961663:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23277568 -> 23281664 00010000:00010000:2.0:1789760732.961680:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0518f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.961693:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0518f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.961702:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0518f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.961710:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760732.961715:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0518f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.961724:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0518f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.961745:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0518f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.961753:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0518f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760732.961855:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493709312:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31134us (31334us total) trans 4294973002 rc 0/0 00000100:00100000:2.0:1789760732.961866:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5713 00000100:00100000:2.0:1789760732.961877:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743232410010 time 743271740682 state 0x2 ref 0 00000100:00100000:1.0:1789760732.964176:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760732.964358:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493709440 00000100:00100000:3.0:1789760732.964368:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493709440 00000100:00100000:3.0:1789760732.964501:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743232410010 time 743271740682 state 0x0 ref 1 00000100:00100000:3.0:1789760732.964526:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743272001098 time 743271740682 state 0x1 ref 1 00000100:00100000:3.0:1789760732.964569:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743272001098 time 743305600017 state 0x0 ref 1 00000100:00100000:3.0:1789760732.964582:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493709440:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760732.964828:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23281664, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:3.0:1789760732.964836:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05196 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.964851:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760732.964855:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.964860:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05196 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.964870:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05196 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760733.016417:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973003 is committed 00002000:00010000:0.0:1789760733.016468:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23281664 -> 23285760 00010000:00010000:0.0:1789760733.016484:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05196 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.016497:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05196 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.016510:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05196 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.016518:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760733.016524:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05196 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.016533:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05196 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.016567:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05196 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.016574:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05196 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760733.016663:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493709440:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52080us (52491us total) trans 4294973003 rc 0/0 00000100:00100000:0.0:1789760733.016674:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5714 00000100:00100000:0.0:1789760733.016684:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743272001098 time 743305600017 state 0x2 ref 0 00000100:00100000:3.0:1789760733.029093:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760733.030320:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493709568 00000100:00100000:1.0:1789760733.030334:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493709568 00000100:00100000:1.0:1789760733.030424:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743272001098 time 743305600017 state 0x0 ref 1 00000100:00100000:1.0:1789760733.030441:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743305860433 time 743305600017 state 0x1 ref 1 00000100:00100000:1.0:1789760733.030474:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743305860433 time 743371509409 state 0x0 ref 1 00000100:00100000:1.0:1789760733.030489:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493709568:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760733.030539:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23285760, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:1.0:1789760733.030557:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0519d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.030578:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:1.0:1789760733.030583:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.030590:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0519d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760733.030731:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789760733.030782:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0519d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.054493:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc5836300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3304:1876699519250944:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760733.054576:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760733.054679:0:6691:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519250944 00000100:00100000:0.0:1789760733.054687:0:6691:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519250944 00000100:00100000:0.0:1789760733.054750:0:6691:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 196 00000100:00100000:0.0:1789760733.054759:0:6691:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519250944:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760733.054783:0:6691:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528023552 pending 1703936 free 3834642432 avail 3824156672 00000100:00080000:1.0:1789760733.055345:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760733.067687:0:6691:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519250944:12345-0@lo:13:osp-pre-0-0.0 Request processed in 12919us (13101us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760733.067701:0:6691:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 196 00000100:00100000:2.0:1789760733.068376:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc5836300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3304:1876699519250944:0@lo:13:osp-pre-0-0.0 00002000:00010000:0.0:1789760733.109827:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23285760 -> 23289856 00010000:00010000:0.0:1789760733.109847:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0519d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.109858:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0519d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.109867:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0519d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.109875:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760733.109880:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0519d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.109889:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0519d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.109906:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0519d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.109915:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0519d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760733.110034:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973004 is committed 00000100:00100000:0.0:1789760733.110173:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493709568:12345-192.168.203.60@tcp:4:dd.0 Request processed in 79684us (81087us total) trans 4294973004 rc 0/0 00000100:00100000:0.0:1789760733.110184:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5715 00000100:00100000:0.0:1789760733.110193:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743305860433 time 743371509409 state 0x2 ref 0 00000100:00100000:1.0:1789760733.113886:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760733.113951:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493709696 00000100:00100000:0.0:1789760733.113988:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493709696 00000100:00100000:0.0:1789760733.114046:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743305860433 time 743371509409 state 0x0 ref 1 00000100:00100000:0.0:1789760733.114059:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743371769825 time 743371509409 state 0x1 ref 1 00000100:00100000:0.0:1789760733.114082:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743371769825 time 743455118950 state 0x0 ref 1 00000100:00100000:0.0:1789760733.114094:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493709696:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760733.114137:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23289856, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:0.0:1789760733.114145:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e051a4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.114160:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760733.114164:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760733.114169:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e051a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.114188:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e051a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760733.143526:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973005 is committed 00002000:00010000:3.0:1789760733.143710:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23289856 -> 23293952 00010000:00010000:3.0:1789760733.143725:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e051a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.143738:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e051a4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.143747:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e051a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.143755:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760733.143760:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e051a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.143770:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e051a4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.143795:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e051a4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.143802:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e051a4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.143894:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493709696:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29797us (30012us total) trans 4294973005 rc 0/0 00000100:00100000:3.0:1789760733.143904:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5716 00000100:00100000:3.0:1789760733.143914:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743371769825 time 743455118950 state 0x2 ref 0 00000100:00100000:1.0:1789760733.159878:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760733.159977:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493709824 00000100:00100000:2.0:1789760733.159988:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493709824 00000100:00100000:2.0:1789760733.160046:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743371769825 time 743455118950 state 0x0 ref 1 00000100:00100000:2.0:1789760733.160061:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743455379366 time 743455118950 state 0x1 ref 1 00000100:00100000:2.0:1789760733.160082:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743455379366 time 743501117718 state 0x0 ref 1 00000100:00100000:2.0:1789760733.160093:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493709824:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760733.160141:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23293952, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:2.0:1789760733.160151:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e051ab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.160167:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760733.160171:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.160178:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e051ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.160197:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e051ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760733.186871:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23293952 -> 23298048 00010000:00010000:0.0:1789760733.186892:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e051ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.186907:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e051ab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.186917:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e051ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.186926:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760733.186931:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e051ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.186941:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e051ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.186979:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e051ab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.186990:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e051ab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760733.187084:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493709824:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26986us (27209us total) trans 4294973006 rc 0/0 00000100:00100000:0.0:1789760733.187095:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5717 00000100:00100000:0.0:1789760733.187105:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743455379366 time 743501117718 state 0x2 ref 0 00000001:00080000:3.0:1789760733.187941:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973006 is committed 00000100:00100000:0.0:1789760733.191379:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760733.191456:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493709952 00000100:00100000:3.0:1789760733.191465:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493709952 00000100:00100000:3.0:1789760733.191518:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743455379366 time 743501117718 state 0x0 ref 1 00000100:00100000:3.0:1789760733.191532:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743501378134 time 743501117718 state 0x1 ref 1 00000100:00100000:3.0:1789760733.191569:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743501378134 time 743532604519 state 0x0 ref 1 00000100:00100000:3.0:1789760733.191581:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493709952:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760733.191722:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23298048, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:3.0:1789760733.191732:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e051b2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.191754:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760733.191759:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.191766:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e051b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.191779:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e051b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760733.253076:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973007 is committed 00002000:00010000:0.0:1789760733.253184:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23298048 -> 23302144 00010000:00010000:0.0:1789760733.253198:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e051b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.253209:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e051b2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.253218:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e051b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.253226:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760733.253231:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e051b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.253240:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e051b2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.253258:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e051b2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.253271:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e051b2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760733.253370:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493709952:12345-192.168.203.60@tcp:4:dd.0 Request processed in 61786us (61995us total) trans 4294973007 rc 0/0 00000100:00100000:0.0:1789760733.253380:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5718 00000100:00100000:0.0:1789760733.253390:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743501378134 time 743532604519 state 0x2 ref 0 00000100:00100000:2.0:1789760733.271198:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760733.272213:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493710080 00000100:00100000:0.0:1789760733.272225:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493710080 00000100:00100000:1.0:1789760733.272772:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743501378134 time 743532604519 state 0x0 ref 1 00000100:00100000:1.0:1789760733.272789:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743532864935 time 743532604519 state 0x1 ref 1 00000100:00100000:1.0:1789760733.272859:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743532864935 time 743613894306 state 0x0 ref 1 00000100:00100000:1.0:1789760733.272874:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493710080:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760733.272921:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23302144, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:1.0:1789760733.272930:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e051b9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.272946:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:1.0:1789760733.272951:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.272981:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e051b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.272996:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e051b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760733.332221:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973008 is committed 00002000:00010000:2.0:1789760733.332447:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23302144 -> 23306240 00010000:00010000:2.0:1789760733.332464:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e051b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.332477:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e051b9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.332486:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e051b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.332494:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760733.332498:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e051b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.332512:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e051b9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.332532:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e051b9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.332551:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e051b9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.332805:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493710080:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59928us (61611us total) trans 4294973008 rc 0/0 00000100:00100000:2.0:1789760733.332817:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5719 00000100:00100000:2.0:1789760733.332828:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743532864935 time 743613894306 state 0x2 ref 0 00000100:00100000:3.0:1789760733.340503:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760733.340709:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493710208 00000100:00100000:0.0:1789760733.340720:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493710208 00000100:00100000:0.0:1789760733.340778:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743532864935 time 743613894306 state 0x0 ref 1 00000100:00100000:0.0:1789760733.340794:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743614154722 time 743613894306 state 0x1 ref 1 00000100:00100000:0.0:1789760733.340858:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743614154722 time 743681893618 state 0x0 ref 1 00000100:00100000:0.0:1789760733.340872:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493710208:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760733.340924:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23306240, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:0.0:1789760733.340934:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e051c0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.340950:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760733.340955:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760733.340992:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e051c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.341006:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e051c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760733.381430:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973009 is committed 00002000:00010000:1.0:1789760733.381694:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23306240 -> 23310336 00010000:00010000:1.0:1789760733.381709:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e051c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.381722:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e051c0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.381732:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e051c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.381744:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760733.381748:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e051c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.381759:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e051c0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.381780:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e051c0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.381787:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e051c0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760733.381885:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493710208:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41011us (41386us total) trans 4294973009 rc 0/0 00000100:00100000:1.0:1789760733.381897:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5720 00000100:00100000:1.0:1789760733.381906:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743614154722 time 743681893618 state 0x2 ref 0 00000100:00100000:0.0:1789760733.386098:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760733.386217:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493710336 00000100:00100000:2.0:1789760733.386226:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493710336 00000100:00100000:2.0:1789760733.386278:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743614154722 time 743681893618 state 0x0 ref 1 00000100:00100000:2.0:1789760733.386292:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743682154034 time 743681893618 state 0x1 ref 1 00000100:00100000:2.0:1789760733.386355:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743682154034 time 743727390326 state 0x0 ref 1 00000100:00100000:2.0:1789760733.386376:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493710336:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760733.386422:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23310336, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:2.0:1789760733.386431:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e051c7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.386461:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760733.386466:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.386472:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e051c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.386484:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e051c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760733.442439:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973010 is committed 00002000:00010000:0.0:1789760733.442691:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23310336 -> 23314432 00010000:00010000:0.0:1789760733.442708:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e051c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.442721:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e051c7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.442730:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e051c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.442738:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760733.442743:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e051c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.442754:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e051c7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.442850:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e051c7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.442864:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e051c7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760733.443505:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493710336:12345-192.168.203.60@tcp:4:dd.0 Request processed in 57124us (57410us total) trans 4294973010 rc 0/0 00000100:00100000:0.0:1789760733.443516:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5721 00000100:00100000:0.0:1789760733.443534:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743682154034 time 743727390326 state 0x2 ref 0 00000100:00100000:2.0:1789760733.454409:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760733.454508:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493710464 00000100:00100000:3.0:1789760733.454517:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493710464 00000100:00100000:3.0:1789760733.454575:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743682154034 time 743727390326 state 0x0 ref 1 00000100:00100000:3.0:1789760733.454640:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743727650742 time 743727390326 state 0x1 ref 1 00000100:00100000:3.0:1789760733.454702:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743727650742 time 743795737612 state 0x0 ref 1 00000100:00100000:3.0:1789760733.454715:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493710464:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760733.454759:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23314432, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:3.0:1789760733.454776:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e051ce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.454792:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760733.454796:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.454802:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e051ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.454817:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e051ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760733.511377:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973011 is committed 00002000:00010000:1.0:1789760733.511521:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23314432 -> 23318528 00010000:00010000:1.0:1789760733.511545:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e051ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.511559:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e051ce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.511568:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e051ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.511576:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760733.511581:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e051ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.511654:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e051ce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.511676:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e051ce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.511689:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e051ce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760733.511878:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493710464:12345-192.168.203.60@tcp:4:dd.0 Request processed in 57161us (57474us total) trans 4294973011 rc 0/0 00000100:00100000:1.0:1789760733.511890:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5722 00000100:00100000:1.0:1789760733.511899:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743727650742 time 743795737612 state 0x2 ref 0 00000100:00100000:1.0:1789760733.515433:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760733.515527:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493710592 00000100:00100000:2.0:1789760733.515598:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493710592 00000100:00100000:2.0:1789760733.515655:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743727650742 time 743795737612 state 0x0 ref 1 00000100:00100000:2.0:1789760733.515670:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743795998028 time 743795737612 state 0x1 ref 1 00000100:00100000:2.0:1789760733.515700:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743795998028 time 743856736091 state 0x0 ref 1 00000100:00100000:2.0:1789760733.515712:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493710592:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760733.515758:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23318528, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:2.0:1789760733.515767:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e051d5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.515783:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760733.515788:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.515795:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e051d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.515810:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e051d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760733.558817:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23318528 -> 23322624 00010000:00010000:1.0:1789760733.558840:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e051d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.558855:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e051d5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.558864:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e051d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.558873:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760733.558878:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e051d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.558886:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e051d5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.558911:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e051d5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.558919:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e051d5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760733.559147:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973012 is committed 00000100:00100000:1.0:1789760733.559295:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493710592:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43579us (43868us total) trans 4294973012 rc 0/0 00000100:00100000:1.0:1789760733.559304:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5723 00000100:00100000:1.0:1789760733.559313:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743795998028 time 743856736091 state 0x2 ref 0 00000100:00100000:0.0:1789760733.568627:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760733.568759:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493710720 00000100:00100000:1.0:1789760733.568769:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493710720 00000100:00100000:1.0:1789760733.568823:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743795998028 time 743856736091 state 0x0 ref 1 00000100:00100000:1.0:1789760733.568838:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743856996507 time 743856736091 state 0x1 ref 1 00000100:00100000:1.0:1789760733.568869:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743856996507 time 743909904376 state 0x0 ref 1 00000100:00100000:1.0:1789760733.568882:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493710720:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760733.568930:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23322624, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:1.0:1789760733.568939:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e051dc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.568953:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:1.0:1789760733.568993:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.569001:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e051dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.569030:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e051dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.569765:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc33ef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3305:1876699519251072:0@lo:400:kworker.0 00000100:00100000:0.0:1789760733.569791:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc33ec380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3303:1876699519251328:0@lo:400:kworker.0 00000100:00100000:3.0:1789760733.569879:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc33edc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3306:1876699519251200:0@lo:400:kworker.0 00000100:00100000:2.0:1789760733.571202:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760733.571278:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc33edf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519251584:0@lo:400:kworker.0 00000100:00100000:2.0:1789760733.571328:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760733.571371:0:13040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519251584 00000100:00100000:1.0:1789760733.571379:0:13040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519251584 00000100:00100000:1.0:1789760733.571441:0:13040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 171 00000100:00100000:1.0:1789760733.571453:0:13040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ead80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519251584:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760733.571472:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760733.571502:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760733.571508:0:13040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ead80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519251584:12345-0@lo:400:kworker.0 Request processed in 58us (179us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760733.571510:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc33eed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3303:1876699519251456:0@lo:400:kworker.0 00000100:00100000:1.0:1789760733.571516:0:13040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 171 00000100:00100000:0.0:1789760733.571535:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1789760733.571549:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789760733.571580:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519251072 00000100:00100000:0.0:1789760733.571584:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519251072 00000100:00100000:0.0:1789760733.571618:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 396 00000100:00100000:0.0:1789760733.571625:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d4e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3305:x1876699519251072:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760733.571656:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d4e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3305:x1876699519251072:12345-0@lo:400:kworker.0 Request processed in 32us (458us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760733.571662:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 396 00000100:00100000:0.0:1789760733.571687:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519251328 00000100:00100000:0.0:1789760733.571691:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519251328 00000100:00100000:0.0:1789760733.571726:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519251456 00000100:00100000:0.0:1789760733.571729:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519251456 00000100:00080000:1.0:1789760733.571731:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789760733.571751:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760733.571753:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 474 00000100:00100000:0.0:1789760733.571760:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876699519251328:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760733.571797:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760733.571801:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876699519251328:12345-0@lo:400:kworker.0 Request processed in 42us (330us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760733.571807:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 474 00000100:00100000:1.0:1789760733.571858:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519251200 00000100:00100000:1.0:1789760733.571863:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519251200 00000100:00100000:1.0:1789760733.571882:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 475 00000100:00100000:1.0:1789760733.571888:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47adc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876699519251456:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760733.571918:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47adc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876699519251456:12345-0@lo:400:kworker.0 Request processed in 31us (382us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760733.571924:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 475 00000100:00100000:1.0:1789760733.571936:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 476 00000100:00100000:1.0:1789760733.571942:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3306:x1876699519251200:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760733.571982:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3306:x1876699519251200:12345-0@lo:400:kworker.0 Request processed in 40us (185us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760733.571989:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 476 00000100:00100000:3.0:1789760733.572218:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc33edc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3306:1876699519251200:0@lo:400:kworker.0 00000100:00100000:2.0:1789760733.572363:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc33edf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519251584:0@lo:400:kworker.0 00000100:00100000:2.0:1789760733.572378:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc33ef480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3305:1876699519251072:0@lo:400:kworker.0 00000100:00100000:2.0:1789760733.579241:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc33eed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3303:1876699519251456:0@lo:400:kworker.0 00000100:00100000:2.0:1789760733.579279:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc33ec380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3303:1876699519251328:0@lo:400:kworker.0 00000001:00080000:3.0:1789760733.616196:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973013 is committed 00002000:00010000:2.0:1789760733.616406:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23322624 -> 23326720 00010000:00010000:2.0:1789760733.616423:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e051dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.616438:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e051dc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.616447:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e051dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.616460:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760733.616465:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e051dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.616475:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e051dc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.616496:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e051dc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.616505:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e051dc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.616692:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493710720:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47807us (48070us total) trans 4294973013 rc 0/0 00000100:00100000:2.0:1789760733.616703:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5724 00000100:00100000:2.0:1789760733.616713:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743856996507 time 743909904376 state 0x2 ref 0 00000100:00100000:0.0:1789760733.620256:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760733.620441:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493710848 00000100:00100000:3.0:1789760733.620450:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493710848 00000100:00100000:3.0:1789760733.620502:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743856996507 time 743909904376 state 0x0 ref 1 00000100:00100000:3.0:1789760733.620589:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743910164792 time 743909904376 state 0x1 ref 1 00000100:00100000:3.0:1789760733.620731:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743910164792 time 743961765925 state 0x0 ref 1 00000100:00100000:3.0:1789760733.620744:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493710848:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760733.620793:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23326720, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xb807 00010000:00010000:3.0:1789760733.620802:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e051e3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.620829:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760733.620835:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.620842:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e051e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.620860:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e051e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760733.651412:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973014 is committed 00002000:00010000:1.0:1789760733.651499:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23326720 -> 23330816 00002000:00010000:1.0:1789760733.651504:0:13038:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760732 -> 1789760733 00002000:00010000:1.0:1789760733.651506:0:13038:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760732 -> 1789760733 00010000:00010000:1.0:1789760733.651519:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e051e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.651533:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e051e3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.651551:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e051e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.651560:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760733.651565:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e051e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.651574:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e051e3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.651653:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e051e3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.651662:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e051e3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760733.651787:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493710848:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31040us (31536us total) trans 4294973014 rc 0/0 00000100:00100000:1.0:1789760733.651796:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5725 00000100:00100000:1.0:1789760733.651806:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743910164792 time 743961765925 state 0x2 ref 0 00000100:00100000:1.0:1789760733.655571:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760733.655652:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493710976 00000100:00100000:3.0:1789760733.655663:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493710976 00000100:00100000:3.0:1789760733.655721:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743910164792 time 743961765925 state 0x0 ref 1 00000100:00100000:3.0:1789760733.655736:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743962026341 time 743961765925 state 0x1 ref 1 00000100:00100000:3.0:1789760733.655765:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743962026341 time 743996801568 state 0x0 ref 1 00000100:00100000:3.0:1789760733.655776:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb07c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493710976:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760733.655824:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23330816, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:3.0:1789760733.655833:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e051ea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.655848:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760733.655852:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.655858:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e051ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.655872:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e051ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760733.693864:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973015 is committed 00002000:00010000:1.0:1789760733.693949:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23330816 -> 23334912 00010000:00010000:1.0:1789760733.694104:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e051ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.694118:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e051ea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.694127:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e051ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.694134:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760733.694139:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e051ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.694148:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e051ea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.694170:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e051ea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.694183:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e051ea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.694490:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb07c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493710976:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38709us (38923us total) trans 4294973015 rc 0/0 00000100:00100000:2.0:1789760733.694501:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5726 00000100:00100000:2.0:1789760733.694511:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743962026341 time 743996801568 state 0x2 ref 0 00000100:00100000:0.0:1789760733.697786:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760733.697873:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493711104 00000100:00100000:1.0:1789760733.697882:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493711104 00000100:00100000:1.0:1789760733.697934:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743962026341 time 743996801568 state 0x0 ref 1 00000100:00100000:1.0:1789760733.697950:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743997061984 time 743996801568 state 0x1 ref 1 00000100:00100000:1.0:1789760733.698118:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743997061984 time 744039151833 state 0x0 ref 1 00000100:00100000:1.0:1789760733.698132:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493711104:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760733.698184:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23334912, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:1.0:1789760733.698194:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e051f1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.698211:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760733.698216:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.698222:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e051f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.698238:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e051f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760733.739177:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973016 is committed 00002000:00010000:2.0:1789760733.739460:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23334912 -> 23339008 00010000:00010000:2.0:1789760733.739476:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e051f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.739490:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e051f1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.739500:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e051f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.739507:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760733.739511:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e051f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.739521:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e051f1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.739548:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e051f1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.739557:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e051f1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.739753:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493711104:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41618us (41970us total) trans 4294973016 rc 0/0 00000100:00100000:2.0:1789760733.739775:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5727 00000100:00100000:2.0:1789760733.739786:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743997061984 time 744039151833 state 0x2 ref 0 00000100:00100000:0.0:1789760733.743894:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760733.744000:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493711232 00000100:00100000:2.0:1789760733.744010:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493711232 00000100:00100000:2.0:1789760733.744190:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 743997061984 time 744039151833 state 0x0 ref 1 00000100:00100000:2.0:1789760733.744205:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744039412249 time 744039151833 state 0x1 ref 1 00000100:00100000:2.0:1789760733.744271:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744039412249 time 744085306629 state 0x0 ref 1 00000100:00100000:2.0:1789760733.744285:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493711232:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760733.744349:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23339008, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:2.0:1789760733.744366:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e051f8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.744381:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760733.744385:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.744392:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e051f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.744405:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e051f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.754832:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760733.772749:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973017 is committed 00002000:00010000:0.0:1789760733.773020:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23339008 -> 23343104 00010000:00010000:0.0:1789760733.773036:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e051f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.773047:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e051f8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.773055:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e051f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.773062:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760733.773066:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e051f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.773073:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e051f8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.773095:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e051f8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.773102:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e051f8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760733.773223:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493711232:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28936us (29334us total) trans 4294973017 rc 0/0 00000100:00100000:0.0:1789760733.773234:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5728 00000100:00100000:0.0:1789760733.773244:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744039412249 time 744085306629 state 0x2 ref 0 00000100:00100000:2.0:1789760733.781908:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760733.782294:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493711360 00000100:00100000:3.0:1789760733.782304:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493711360 00000100:00100000:3.0:1789760733.782357:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744039412249 time 744085306629 state 0x0 ref 1 00000100:00100000:3.0:1789760733.782374:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744085567045 time 744085306629 state 0x1 ref 1 00000100:00100000:3.0:1789760733.782407:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744085567045 time 744123442612 state 0x0 ref 1 00000100:00100000:3.0:1789760733.782420:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493711360:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760733.782468:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23343104, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:3.0:1789760733.782477:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e051ff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.782492:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760733.782496:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.782502:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e051ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.782515:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e051ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760733.834735:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973018 is committed 00002000:00010000:0.0:1789760733.834819:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23343104 -> 23347200 00010000:00010000:0.0:1789760733.834834:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e051ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.834843:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e051ff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.834849:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e051ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.834853:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760733.834857:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e051ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.834867:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e051ff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.834882:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e051ff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.834886:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e051ff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760733.835131:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493711360:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52702us (53220us total) trans 4294973018 rc 0/0 00000100:00100000:0.0:1789760733.835141:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5729 00000100:00100000:0.0:1789760733.835151:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744085567045 time 744123442612 state 0x2 ref 0 00000100:00100000:1.0:1789760733.844718:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760733.844948:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493711488 00000100:00100000:0.0:1789760733.844958:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493711488 00000100:00100000:0.0:1789760733.845269:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744085567045 time 744123442612 state 0x0 ref 1 00000100:00100000:0.0:1789760733.845286:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744123703028 time 744123442612 state 0x1 ref 1 00000100:00100000:0.0:1789760733.845322:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744123703028 time 744186357263 state 0x0 ref 1 00000100:00100000:0.0:1789760733.845334:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493711488:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760733.845387:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23347200, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:0.0:1789760733.845396:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05206 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.845418:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760733.845423:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760733.845429:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05206 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.845442:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05206 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760733.881499:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973019 is committed 00002000:00010000:2.0:1789760733.881509:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23347200 -> 23351296 00010000:00010000:2.0:1789760733.881523:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05206 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.881546:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05206 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.881556:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05206 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.881563:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760733.881568:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05206 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.881576:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05206 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.881595:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05206 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.881603:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05206 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.881697:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493711488:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36359us (36985us total) trans 4294973019 rc 0/0 00000100:00100000:2.0:1789760733.881707:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5730 00000100:00100000:2.0:1789760733.881716:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744123703028 time 744186357263 state 0x2 ref 0 00000100:00100000:3.0:1789760733.894171:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760733.894365:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493711616 00000100:00100000:2.0:1789760733.894376:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493711616 00000100:00100000:2.0:1789760733.894429:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744123703028 time 744186357263 state 0x0 ref 1 00000100:00100000:2.0:1789760733.894445:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744186617679 time 744186357263 state 0x1 ref 1 00000100:00100000:2.0:1789760733.894478:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744186617679 time 744235513509 state 0x0 ref 1 00000100:00100000:2.0:1789760733.894490:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493711616:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760733.894560:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23351296, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:2.0:1789760733.894569:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0520d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.894582:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760733.894597:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.894604:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0520d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.894617:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0520d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760733.954385:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973020 is committed 00002000:00010000:1.0:1789760733.954451:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23351296 -> 23355392 00010000:00010000:1.0:1789760733.954465:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0520d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.954479:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0520d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.954487:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0520d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.954495:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760733.954499:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0520d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.954508:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0520d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.954545:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0520d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.954553:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0520d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760733.955338:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493711616:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60464us (60792us total) trans 4294973020 rc 0/0 00000100:00100000:1.0:1789760733.955353:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5731 00000100:00100000:1.0:1789760733.955363:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744186617679 time 744235513509 state 0x2 ref 0 00000100:00100000:3.0:1789760733.963135:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760733.963260:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493711744 00000100:00100000:2.0:1789760733.963274:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493711744 00000100:00100000:2.0:1789760733.963338:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744186617679 time 744235513509 state 0x0 ref 1 00000100:00100000:2.0:1789760733.963354:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744235773925 time 744235513509 state 0x1 ref 1 00000100:00100000:2.0:1789760733.963396:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744235773925 time 744304431278 state 0x0 ref 1 00000100:00100000:2.0:1789760733.963413:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493711744:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760733.963464:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23355392, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:2.0:1789760733.963472:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05214 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.963488:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760733.963493:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.963500:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05214 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.963514:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05214 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760734.025227:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973021 is committed 00002000:00010000:0.0:1789760734.025485:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23355392 -> 23359488 00010000:00010000:0.0:1789760734.025506:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05214 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.025522:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05214 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.025542:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05214 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.025551:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760734.025557:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05214 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.025568:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05214 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.025602:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05214 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.025611:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05214 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760734.025731:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493711744:12345-192.168.203.60@tcp:4:dd.0 Request processed in 62313us (62604us total) trans 4294973021 rc 0/0 00000100:00100000:0.0:1789760734.025744:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5732 00000100:00100000:0.0:1789760734.025756:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744235773925 time 744304431278 state 0x2 ref 0 00000100:00100000:0.0:1789760734.033258:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760734.033370:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493711872 00000100:00100000:3.0:1789760734.033383:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493711872 00000100:00100000:3.0:1789760734.033461:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744235773925 time 744304431278 state 0x0 ref 1 00000100:00100000:3.0:1789760734.033480:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744304691694 time 744304431278 state 0x1 ref 1 00000100:00100000:3.0:1789760734.033522:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744304691694 time 744374556874 state 0x0 ref 1 00000100:00100000:3.0:1789760734.033544:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493711872:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760734.033618:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23359488, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:3.0:1789760734.033630:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0521b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.033650:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760734.033655:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.033663:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0521b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.033680:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0521b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760734.075945:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973022 is committed 00002000:00010000:3.0:1789760734.076175:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23359488 -> 23363584 00010000:00010000:3.0:1789760734.076195:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0521b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.076210:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0521b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.076220:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0521b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.076229:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760734.076235:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0521b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.076246:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0521b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.076271:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0521b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.076280:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0521b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.076401:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493711872:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42862us (43151us total) trans 4294973022 rc 0/0 00000100:00100000:3.0:1789760734.076414:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5733 00000100:00100000:3.0:1789760734.076428:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744304691694 time 744374556874 state 0x2 ref 0 00000100:00100000:3.0:1789760734.080607:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc452aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519251712:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760734.080696:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760734.080943:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519251712 00000100:00100000:2.0:1789760734.080954:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519251712 00000100:00100000:2.0:1789760734.081045:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 197 00000100:00100000:2.0:1789760734.081057:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519251712:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760734.081108:0:6690:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760734.082084:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519251712:12345-0@lo:13:osp-pre-1-0.0 Request processed in 1026us (1385us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760734.082096:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 197 00000100:00100000:3.0:1789760734.085865:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc452aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519251712:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760734.088195:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760734.090158:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493712000 00000100:00100000:2.0:1789760734.090177:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493712000 00000100:00100000:2.0:1789760734.090265:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744304691694 time 744374556874 state 0x0 ref 1 00000100:00100000:2.0:1789760734.090287:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744374817290 time 744374556874 state 0x1 ref 1 00000100:00100000:2.0:1789760734.090377:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744374817290 time 744431410949 state 0x0 ref 1 00000100:00100000:2.0:1789760734.090392:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493712000:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760734.090463:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23363584, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:2.0:1789760734.090474:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05222 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.090494:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760734.090498:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.090505:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05222 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.090522:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05222 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760734.143187:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23363584 -> 23367680 00010000:00010000:2.0:1789760734.143221:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05222 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.143237:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05222 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.143249:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05222 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.143264:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760734.143270:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05222 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.143280:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05222 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.143313:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05222 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.143322:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05222 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.143463:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493712000:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53066us (55273us total) trans 4294973023 rc 0/0 00000100:00100000:2.0:1789760734.143477:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5734 00000100:00100000:2.0:1789760734.143491:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744374817290 time 744431410949 state 0x2 ref 0 00000001:00080000:0.0:1789760734.144728:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973023 is committed 00000100:00100000:0.0:1789760734.148422:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760734.148529:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493712128 00000100:00100000:1.0:1789760734.148552:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493712128 00000100:00100000:1.0:1789760734.148695:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744374817290 time 744431410949 state 0x0 ref 1 00000100:00100000:1.0:1789760734.148713:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744431671365 time 744431410949 state 0x1 ref 1 00000100:00100000:1.0:1789760734.148783:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744431671365 time 744489818323 state 0x0 ref 1 00000100:00100000:1.0:1789760734.148799:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493712128:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760734.148870:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23367680, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:1.0:1789760734.148882:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05229 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.148901:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760734.148906:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.148915:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05229 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.148931:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05229 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760734.203047:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973024 is committed 00002000:00010000:1.0:1789760734.203261:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23367680 -> 23371776 00010000:00010000:1.0:1789760734.203282:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05229 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.203299:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05229 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.203309:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05229 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.203317:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760734.203322:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05229 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.203332:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05229 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.203355:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05229 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.203362:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05229 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.203486:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493712128:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54683us (55075us total) trans 4294973024 rc 0/0 00000100:00100000:1.0:1789760734.203498:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5735 00000100:00100000:1.0:1789760734.203509:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744431671365 time 744489818323 state 0x2 ref 0 00000100:00100000:1.0:1789760734.209571:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760734.209677:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493712256 00000100:00100000:0.0:1789760734.209692:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493712256 00000100:00100000:0.0:1789760734.209768:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744431671365 time 744489818323 state 0x0 ref 1 00000100:00100000:0.0:1789760734.209789:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744490078739 time 744489818323 state 0x1 ref 1 00000100:00100000:0.0:1789760734.209888:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744490078739 time 744550921319 state 0x0 ref 1 00000100:00100000:0.0:1789760734.209904:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493712256:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760734.210011:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23371776, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:0.0:1789760734.210023:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05230 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.210051:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760734.210057:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760734.210065:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05230 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.210081:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05230 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760734.263902:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973025 is committed 00002000:00010000:2.0:1789760734.264045:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23371776 -> 23375872 00010000:00010000:2.0:1789760734.264065:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05230 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.264080:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05230 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.264097:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05230 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.264105:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760734.264111:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05230 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.264121:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05230 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.264145:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05230 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.264153:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05230 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.264279:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493712256:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54370us (54716us total) trans 4294973025 rc 0/0 00000100:00100000:2.0:1789760734.264293:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5736 00000100:00100000:2.0:1789760734.264305:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744490078739 time 744550921319 state 0x2 ref 0 00000100:00100000:2.0:1789760734.267947:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760734.268396:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493712384 00000100:00100000:3.0:1789760734.268405:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493712384 00000100:00100000:3.0:1789760734.268459:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744490078739 time 744550921319 state 0x0 ref 1 00000100:00100000:3.0:1789760734.268472:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744551181735 time 744550921319 state 0x1 ref 1 00000100:00100000:3.0:1789760734.268498:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744551181735 time 744609534350 state 0x0 ref 1 00000100:00100000:3.0:1789760734.268511:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493712384:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760734.268567:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23375872, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:3.0:1789760734.268576:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05237 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.268656:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760734.268661:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.268668:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05237 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.268683:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05237 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760734.302457:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973026 is committed 00002000:00010000:2.0:1789760734.302624:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23375872 -> 23379968 00010000:00010000:2.0:1789760734.302638:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05237 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.302656:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05237 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.302666:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05237 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.302673:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760734.302678:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05237 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.302688:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05237 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.302708:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05237 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.302715:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05237 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.302801:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493712384:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34286us (34862us total) trans 4294973026 rc 0/0 00000100:00100000:2.0:1789760734.302811:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5737 00000100:00100000:2.0:1789760734.302821:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744551181735 time 744609534350 state 0x2 ref 0 00000100:00100000:1.0:1789760734.307718:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760734.307818:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493712512 00000100:00100000:2.0:1789760734.307830:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493712512 00000100:00100000:2.0:1789760734.307885:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744551181735 time 744609534350 state 0x0 ref 1 00000100:00100000:2.0:1789760734.307899:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744609794766 time 744609534350 state 0x1 ref 1 00000100:00100000:2.0:1789760734.307988:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744609794766 time 744649023394 state 0x0 ref 1 00000100:00100000:2.0:1789760734.308000:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493712512:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760734.308172:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23379968, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:2.0:1789760734.308182:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0523e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.308196:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760734.308200:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.308207:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0523e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.308220:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0523e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760734.343000:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973027 is committed 00002000:00010000:2.0:1789760734.343244:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23379968 -> 23384064 00010000:00010000:2.0:1789760734.343259:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0523e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.343271:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0523e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.343287:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0523e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.343294:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760734.343299:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0523e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.343308:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0523e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.343327:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0523e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.343335:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0523e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.343426:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493712512:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35422us (35712us total) trans 4294973027 rc 0/0 00000100:00100000:2.0:1789760734.343436:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5738 00000100:00100000:2.0:1789760734.343445:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744609794766 time 744649023394 state 0x2 ref 0 00000100:00100000:1.0:1789760734.346648:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760734.346745:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493712640 00000100:00100000:2.0:1789760734.346756:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493712640 00000100:00100000:2.0:1789760734.346813:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744609794766 time 744649023394 state 0x0 ref 1 00000100:00100000:2.0:1789760734.346828:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744649283810 time 744649023394 state 0x1 ref 1 00000100:00100000:2.0:1789760734.346891:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744649283810 time 744687926570 state 0x0 ref 1 00000100:00100000:2.0:1789760734.346904:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493712640:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760734.346950:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23384064, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:2.0:1789760734.346989:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05245 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.347004:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760734.347009:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.347016:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05245 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.347029:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05245 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760734.387550:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23384064 -> 23388160 00010000:00010000:3.0:1789760734.387579:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05245 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.387592:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05245 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.387601:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05245 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.387609:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760734.387622:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05245 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.387632:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05245 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.387651:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05245 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.387658:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05245 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.387754:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493712640:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40848us (41110us total) trans 4294973028 rc 0/0 00000100:00100000:3.0:1789760734.387763:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5739 00000100:00100000:3.0:1789760734.387773:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744649283810 time 744687926570 state 0x2 ref 0 00000001:00080000:3.0:1789760734.387810:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973028 is committed 00000100:00100000:1.0:1789760734.396806:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760734.396891:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493712768 00000100:00100000:2.0:1789760734.396900:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493712768 00000100:00100000:2.0:1789760734.396986:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744649283810 time 744687926570 state 0x0 ref 1 00000100:00100000:2.0:1789760734.397003:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744688186986 time 744687926570 state 0x1 ref 1 00000100:00100000:2.0:1789760734.397027:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744688186986 time 744738063641 state 0x0 ref 1 00000100:00100000:2.0:1789760734.397326:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7aa7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493712768:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760734.397378:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23388160, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:2.0:1789760734.397388:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0524c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.397402:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760734.397406:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.397412:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0524c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.397425:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0524c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760734.451712:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973029 is committed 00002000:00010000:0.0:1789760734.451766:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23388160 -> 23392256 00010000:00010000:0.0:1789760734.451783:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0524c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.451797:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0524c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.451806:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0524c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.451815:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760734.451820:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0524c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.451829:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0524c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.451854:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0524c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.451863:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0524c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760734.451951:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7aa7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493712768:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54908us (55149us total) trans 4294973029 rc 0/0 00000100:00100000:0.0:1789760734.452005:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5740 00000100:00100000:0.0:1789760734.452016:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744688186986 time 744738063641 state 0x2 ref 0 00000100:00100000:1.0:1789760734.456440:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760734.456520:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493712896 00000100:00100000:3.0:1789760734.456543:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493712896 00000100:00100000:3.0:1789760734.456606:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744688186986 time 744738063641 state 0x0 ref 1 00000100:00100000:3.0:1789760734.456621:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744738324057 time 744738063641 state 0x1 ref 1 00000100:00100000:3.0:1789760734.456650:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744738324057 time 744797685651 state 0x0 ref 1 00000100:00100000:3.0:1789760734.456662:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8555180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493712896:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760734.456713:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23392256, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:3.0:1789760734.456722:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05253 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.456737:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760734.456741:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.456747:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05253 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.456761:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05253 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760734.523833:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973030 is committed 00002000:00010000:3.0:1789760734.523944:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23392256 -> 23396352 00010000:00010000:3.0:1789760734.523984:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05253 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.523998:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05253 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.524008:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05253 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.524015:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760734.524020:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05253 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.524030:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05253 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.524050:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05253 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.524059:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05253 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.524148:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8555180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493712896:12345-192.168.203.60@tcp:4:dd.0 Request processed in 67483us (67714us total) trans 4294973030 rc 0/0 00000100:00100000:3.0:1789760734.524158:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5741 00000100:00100000:3.0:1789760734.524167:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744738324057 time 744797685651 state 0x2 ref 0 00000100:00100000:1.0:1789760734.527703:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760734.527820:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493713024 00000100:00100000:0.0:1789760734.527832:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493713024 00000100:00100000:0.0:1789760734.527890:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744738324057 time 744797685651 state 0x0 ref 1 00000100:00100000:0.0:1789760734.527917:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744797946067 time 744797685651 state 0x1 ref 1 00000100:00100000:0.0:1789760734.527952:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744797946067 time 744868987536 state 0x0 ref 1 00000100:00100000:0.0:1789760734.528002:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493713024:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760734.528053:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23396352, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:0.0:1789760734.528061:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0525a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.528077:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760734.528082:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760734.528088:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0525a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.528102:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0525a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760734.555611:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973031 is committed 00002000:00010000:0.0:1789760734.555764:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23396352 -> 23400448 00010000:00010000:0.0:1789760734.555779:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0525a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.555791:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0525a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.555800:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0525a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.555808:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760734.555812:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0525a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.555821:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0525a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.555838:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0525a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.555846:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0525a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760734.556103:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493713024:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28099us (28406us total) trans 4294973031 rc 0/0 00000100:00100000:0.0:1789760734.556115:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5742 00000100:00100000:0.0:1789760734.556125:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744797946067 time 744868987536 state 0x2 ref 0 00000100:00100000:2.0:1789760734.560696:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760734.560817:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493713152 00000100:00100000:3.0:1789760734.560827:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493713152 00000100:00100000:3.0:1789760734.560882:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744797946067 time 744868987536 state 0x0 ref 1 00000100:00100000:3.0:1789760734.560899:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744869247952 time 744868987536 state 0x1 ref 1 00000100:00100000:3.0:1789760734.560932:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744869247952 time 744901967200 state 0x0 ref 1 00000100:00100000:3.0:1789760734.560946:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493713152:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760734.561021:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23400448, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:3.0:1789760734.561031:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05261 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.561046:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760734.561051:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.561058:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05261 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.561071:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05261 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760734.595062:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973032 is committed 00002000:00010000:3.0:1789760734.595171:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23400448 -> 23404544 00010000:00010000:3.0:1789760734.595187:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05261 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.595199:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05261 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.595209:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05261 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.595216:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760734.595221:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05261 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.595230:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05261 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.595250:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05261 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.595259:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05261 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.595351:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493713152:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34404us (34670us total) trans 4294973032 rc 0/0 00000100:00100000:3.0:1789760734.595362:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5743 00000100:00100000:3.0:1789760734.595371:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744869247952 time 744901967200 state 0x2 ref 0 00000100:00100000:1.0:1789760734.601705:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760734.601797:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493713280 00000100:00100000:2.0:1789760734.601808:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493713280 00000100:00100000:2.0:1789760734.601865:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744869247952 time 744901967200 state 0x0 ref 1 00000100:00100000:2.0:1789760734.601881:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744902227616 time 744901967200 state 0x1 ref 1 00000100:00100000:2.0:1789760734.601910:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744902227616 time 744942946141 state 0x0 ref 1 00000100:00100000:2.0:1789760734.601921:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8556a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493713280:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760734.602125:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23404544, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xb807 00010000:00010000:2.0:1789760734.602135:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05268 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.602153:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760734.602156:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.602161:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05268 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.602174:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05268 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760734.631122:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23404544 -> 23408640 00002000:00010000:2.0:1789760734.631130:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760733 -> 1789760734 00002000:00010000:2.0:1789760734.631133:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760733 -> 1789760734 00010000:00010000:2.0:1789760734.631147:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05268 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.631160:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05268 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.631169:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05268 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.631176:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760734.631181:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05268 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.631189:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05268 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.631208:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05268 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.631215:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05268 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.631300:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8556a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493713280:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29375us (29600us total) trans 4294973033 rc 0/0 00000100:00100000:2.0:1789760734.631310:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5744 00000100:00100000:2.0:1789760734.631320:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744902227616 time 744942946141 state 0x2 ref 0 00000001:00080000:2.0:1789760734.631358:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973033 is committed 00000100:00100000:1.0:1789760734.634360:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760734.634444:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493713408 00000100:00100000:2.0:1789760734.634454:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493713408 00000100:00100000:2.0:1789760734.634512:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744902227616 time 744942946141 state 0x0 ref 1 00000100:00100000:2.0:1789760734.634538:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744943206557 time 744942946141 state 0x1 ref 1 00000100:00100000:2.0:1789760734.634564:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744943206557 time 744975599703 state 0x0 ref 1 00000100:00100000:2.0:1789760734.634577:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493713408:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760734.634638:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23408640, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:2.0:1789760734.634648:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0526f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.634672:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760734.634677:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.634683:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0526f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.634697:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0526f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760734.668998:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23408640 -> 23412736 00010000:00010000:2.0:1789760734.669028:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0526f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.669042:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0526f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.669051:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0526f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.669059:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760734.669063:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0526f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.669072:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0526f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.669106:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0526f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.669115:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0526f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.669200:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493713408:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34620us (34845us total) trans 4294973034 rc 0/0 00000100:00100000:2.0:1789760734.669210:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5745 00000100:00100000:2.0:1789760734.669219:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744943206557 time 744975599703 state 0x2 ref 0 00000001:00080000:0.0:1789760734.670223:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973034 is committed 00000100:00100000:1.0:1789760734.678828:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760734.678935:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493713536 00000100:00100000:0.0:1789760734.678946:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493713536 00000100:00100000:0.0:1789760734.679050:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744943206557 time 744975599703 state 0x0 ref 1 00000100:00100000:0.0:1789760734.679066:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744975860119 time 744975599703 state 0x1 ref 1 00000100:00100000:0.0:1789760734.679102:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744975860119 time 745020137844 state 0x0 ref 1 00000100:00100000:0.0:1789760734.679115:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493713536:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760734.679162:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23412736, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:0.0:1789760734.679172:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05276 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.679189:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:0.0:1789760734.679194:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760734.679201:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05276 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.679215:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05276 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760734.709369:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973035 is committed 00002000:00010000:1.0:1789760734.709448:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23412736 -> 23416832 00010000:00010000:1.0:1789760734.709462:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05276 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.709475:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05276 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.709483:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05276 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.709490:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760734.709495:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05276 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.709504:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05276 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.709524:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05276 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.709546:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05276 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.709734:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493713536:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30616us (30912us total) trans 4294973035 rc 0/0 00000100:00100000:1.0:1789760734.709745:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5746 00000100:00100000:1.0:1789760734.709755:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744975860119 time 745020137844 state 0x2 ref 0 00000100:00100000:1.0:1789760734.719375:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760734.719608:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493713664 00000100:00100000:3.0:1789760734.719627:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493713664 00000100:00100000:3.0:1789760734.719703:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 744975860119 time 745020137844 state 0x0 ref 1 00000100:00100000:3.0:1789760734.719722:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745020398260 time 745020137844 state 0x1 ref 1 00000100:00100000:3.0:1789760734.719758:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745020398260 time 745060792438 state 0x0 ref 1 00000100:00100000:3.0:1789760734.719772:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493713664:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760734.724596:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23416832, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:3.0:1789760734.724621:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0527d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.724643:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760734.724648:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.724655:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0527d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.724671:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0527d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.781674:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760734.795906:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973036 is committed 00002000:00010000:0.0:1789760734.796158:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23416832 -> 23420928 00010000:00010000:0.0:1789760734.796180:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0527d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.796195:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0527d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.796205:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0527d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.796213:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760734.796218:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0527d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.796228:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0527d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.796255:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0527d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.796264:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0527d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760734.796392:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493713664:12345-192.168.203.60@tcp:4:dd.0 Request processed in 76614us (77019us total) trans 4294973036 rc 0/0 00000100:00100000:0.0:1789760734.796407:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5747 00000100:00100000:0.0:1789760734.796422:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745020398260 time 745060792438 state 0x2 ref 0 00000100:00100000:0.0:1789760734.799093:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760734.800366:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493713792 00000100:00100000:2.0:1789760734.800384:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493713792 00000100:00100000:3.0:1789760734.800728:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745020398260 time 745060792438 state 0x0 ref 1 00000100:00100000:3.0:1789760734.800749:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745061052854 time 745060792438 state 0x1 ref 1 00000100:00100000:3.0:1789760734.800842:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745061052854 time 745141876690 state 0x0 ref 1 00000100:00100000:3.0:1789760734.800858:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493713792:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760734.800928:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23420928, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:3.0:1789760734.800939:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05284 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.800995:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760734.801001:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.801019:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05284 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.801037:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05284 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760734.871003:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973037 is committed 00002000:00010000:3.0:1789760734.871431:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23420928 -> 23425024 00010000:00010000:3.0:1789760734.871450:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05284 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.871466:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05284 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.871477:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05284 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.871486:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760734.871492:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05284 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.871501:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05284 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.871644:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05284 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.871655:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05284 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.871780:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493713792:12345-192.168.203.60@tcp:4:dd.0 Request processed in 70918us (72695us total) trans 4294973037 rc 0/0 00000100:00100000:3.0:1789760734.872161:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5748 00000100:00100000:3.0:1789760734.872176:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745061052854 time 745141876690 state 0x2 ref 0 00000100:00100000:1.0:1789760734.874679:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760734.874847:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493713920 00000100:00100000:2.0:1789760734.874860:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493713920 00000100:00100000:2.0:1789760734.874936:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745061052854 time 745141876690 state 0x0 ref 1 00000100:00100000:2.0:1789760734.874955:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745142137106 time 745141876690 state 0x1 ref 1 00000100:00100000:2.0:1789760734.875018:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745142137106 time 745216052933 state 0x0 ref 1 00000100:00100000:2.0:1789760734.875034:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493713920:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760734.875098:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23425024, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:2.0:1789760734.875109:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0528b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.875129:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760734.875134:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.875143:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0528b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.875159:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0528b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760734.914372:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973038 is committed 00002000:00010000:2.0:1789760734.915691:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23425024 -> 23429120 00010000:00010000:2.0:1789760734.915716:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0528b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.915732:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0528b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.915742:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0528b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.915751:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760734.915756:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0528b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.915766:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0528b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.915792:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0528b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.915800:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0528b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.916011:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493713920:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40973us (41342us total) trans 4294973038 rc 0/0 00000100:00100000:2.0:1789760734.916024:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5749 00000100:00100000:2.0:1789760734.916037:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745142137106 time 745216052933 state 0x2 ref 0 00000100:00100000:1.0:1789760734.926732:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760734.926873:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493714048 00000100:00100000:0.0:1789760734.927027:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493714048 00000100:00100000:0.0:1789760734.927108:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745142137106 time 745216052933 state 0x0 ref 1 00000100:00100000:0.0:1789760734.927128:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745216313349 time 745216052933 state 0x1 ref 1 00000100:00100000:0.0:1789760734.927216:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745216313349 time 745268250651 state 0x0 ref 1 00000100:00100000:0.0:1789760734.927231:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493714048:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760734.927301:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23429120, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:0.0:1789760734.927312:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05292 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.927332:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:0.0:1789760734.927338:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760734.927346:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05292 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.927363:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05292 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760735.025605:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23429120 -> 23433216 00010000:00010000:2.0:1789760735.025641:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05292 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.025657:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05292 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.025668:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05292 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.025677:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760735.025683:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05292 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.025695:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05292 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.025719:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05292 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.025734:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05292 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760735.025856:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493714048:12345-192.168.203.60@tcp:4:dd.0 Request processed in 98621us (99132us total) trans 4294973039 rc 0/0 00000100:00100000:2.0:1789760735.025869:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5750 00000100:00100000:2.0:1789760735.025882:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745216313349 time 745268250651 state 0x2 ref 0 00000001:00080000:3.0:1789760735.025945:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973039 is committed 00000100:00100000:0.0:1789760735.031708:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760735.036785:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493714176 00000100:00100000:3.0:1789760735.036807:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493714176 00000100:00080000:1.0:1789760735.036912:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760735.036916:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745216313349 time 745268250651 state 0x0 ref 1 00000100:00100000:3.0:1789760735.036943:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745268511067 time 745268250651 state 0x1 ref 1 00000100:00100000:3.0:1789760735.036994:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745268511067 time 745378028970 state 0x0 ref 1 00000100:00100000:3.0:1789760735.037013:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493714176:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760735.037090:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23433216, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:3.0:1789760735.037103:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05299 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.037523:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760735.037538:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.037547:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05299 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.037678:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05299 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760735.108417:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973040 is committed 00002000:00010000:0.0:1789760735.108612:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23433216 -> 23437312 00010000:00010000:0.0:1789760735.108632:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05299 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.108646:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05299 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.108655:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05299 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.108664:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760735.108670:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05299 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.108681:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05299 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.108706:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05299 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.108715:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05299 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760735.108830:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493714176:12345-192.168.203.60@tcp:4:dd.0 Request processed in 71815us (77133us total) trans 4294973040 rc 0/0 00000100:00100000:0.0:1789760735.108842:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5751 00000100:00100000:0.0:1789760735.108855:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745268511067 time 745378028970 state 0x2 ref 0 00000100:00100000:1.0:1789760735.118460:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760735.118695:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493714304 00000100:00100000:3.0:1789760735.118712:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493714304 00000100:00100000:3.0:1789760735.118800:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745268511067 time 745378028970 state 0x0 ref 1 00000100:00100000:3.0:1789760735.118821:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745378289386 time 745378028970 state 0x1 ref 1 00000100:00100000:3.0:1789760735.118858:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745378289386 time 745459893285 state 0x0 ref 1 00000100:00100000:3.0:1789760735.118877:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493714304:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760735.118955:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23437312, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:3.0:1789760735.119215:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052a0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.119235:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760735.119240:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.119247:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.119507:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760735.155234:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23437312 -> 23441408 00010000:00010000:3.0:1789760735.155263:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.155279:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052a0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.155289:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.155297:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760735.155303:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.155314:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052a0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.155339:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052a0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.155347:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052a0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.155466:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493714304:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36584us (37016us total) trans 4294973041 rc 0/0 00000100:00100000:3.0:1789760735.155478:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5752 00000100:00100000:3.0:1789760735.155491:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745378289386 time 745459893285 state 0x2 ref 0 00000001:00080000:1.0:1789760735.156227:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973041 is committed 00000100:00100000:1.0:1789760735.164599:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760735.164837:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493714432 00000100:00100000:2.0:1789760735.164851:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493714432 00000100:00100000:2.0:1789760735.164931:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745378289386 time 745459893285 state 0x0 ref 1 00000100:00100000:2.0:1789760735.164951:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745460153701 time 745459893285 state 0x1 ref 1 00000100:00100000:0.0:1789760735.165102:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745460153701 time 745506133541 state 0x0 ref 1 00000100:00100000:0.0:1789760735.165120:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493714432:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760735.165187:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23441408, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:0.0:1789760735.165199:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e052a7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.165217:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:0.0:1789760735.165223:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760735.165230:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e052a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.165246:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e052a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760735.213951:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23441408 -> 23445504 00010000:00010000:2.0:1789760735.214002:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e052a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.214019:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e052a7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.214030:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e052a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.214039:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760735.214046:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e052a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.214058:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e052a7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.214084:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e052a7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.214092:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e052a7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760735.214222:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493714432:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49099us (49634us total) trans 4294973042 rc 0/0 00000100:00100000:2.0:1789760735.214235:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5753 00000100:00100000:2.0:1789760735.214248:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745460153701 time 745506133541 state 0x2 ref 0 00000001:00080000:1.0:1789760735.215446:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973042 is committed 00000100:00100000:0.0:1789760735.220945:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760735.221320:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493714560 00000100:00100000:2.0:1789760735.221333:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493714560 00000100:00100000:2.0:1789760735.221487:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745460153701 time 745506133541 state 0x0 ref 1 00000100:00100000:2.0:1789760735.221815:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745506393957 time 745506133541 state 0x1 ref 1 00000100:00100000:2.0:1789760735.221901:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745506393957 time 745562935119 state 0x0 ref 1 00000100:00100000:2.0:1789760735.221917:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493714560:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760735.222007:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23445504, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:2.0:1789760735.222018:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e052ae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.222038:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760735.222044:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.222052:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e052ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.222067:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e052ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760735.313773:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973043 is committed 00002000:00010000:3.0:1789760735.314005:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23445504 -> 23449600 00010000:00010000:3.0:1789760735.314027:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e052ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.314044:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e052ae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.314055:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e052ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.314064:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760735.314071:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e052ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.314083:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e052ae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.314110:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e052ae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.314119:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e052ae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.314293:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493714560:12345-192.168.203.60@tcp:4:dd.0 Request processed in 92372us (93360us total) trans 4294973043 rc 0/0 00000100:00100000:3.0:1789760735.314306:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5754 00000100:00100000:3.0:1789760735.314319:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745506393957 time 745562935119 state 0x2 ref 0 00000100:00100000:3.0:1789760735.321078:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760735.321263:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493714688 00000100:00100000:2.0:1789760735.321280:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493714688 00000100:00100000:2.0:1789760735.321378:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745506393957 time 745562935119 state 0x0 ref 1 00000100:00100000:2.0:1789760735.321402:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745563195535 time 745562935119 state 0x1 ref 1 00000100:00100000:2.0:1789760735.321503:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745563195535 time 745662536357 state 0x0 ref 1 00000100:00100000:2.0:1789760735.321521:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493714688:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760735.321683:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23449600, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:2.0:1789760735.321694:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e052b5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.321712:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760735.321718:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.321726:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e052b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.321750:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e052b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760735.426988:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973044 is committed 00002000:00010000:3.0:1789760735.427264:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23449600 -> 23453696 00010000:00010000:3.0:1789760735.427286:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e052b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.427301:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e052b5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.427311:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e052b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.427327:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760735.427333:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e052b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.427344:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e052b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.427370:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e052b5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.427379:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e052b5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.427513:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493714688:12345-192.168.203.60@tcp:4:dd.0 Request processed in 105988us (106442us total) trans 4294973044 rc 0/0 00000100:00100000:3.0:1789760735.427533:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5755 00000100:00100000:3.0:1789760735.427547:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745563195535 time 745662536357 state 0x2 ref 0 00000100:00100000:1.0:1789760735.431758:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760735.431900:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493714816 00000100:00100000:3.0:1789760735.431913:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493714816 00000100:00100000:3.0:1789760735.432043:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745563195535 time 745662536357 state 0x0 ref 1 00000100:00100000:3.0:1789760735.432063:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745662796773 time 745662536357 state 0x1 ref 1 00000100:00100000:3.0:1789760735.432094:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745662796773 time 745773128357 state 0x0 ref 1 00000100:00100000:3.0:1789760735.432113:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc17480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493714816:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760735.432188:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23453696, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:3.0:1789760735.432200:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052bc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.432220:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760735.432225:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.432232:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.432248:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760735.587106:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973045 is committed 00002000:00010000:3.0:1789760735.587515:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23453696 -> 23457792 00010000:00010000:3.0:1789760735.587572:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.587588:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052bc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.587598:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.587605:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760735.587611:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.587622:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052bc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.587647:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052bc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.587657:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052bc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.587792:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc17480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493714816:12345-192.168.203.60@tcp:4:dd.0 Request processed in 155675us (156045us total) trans 4294973045 rc 0/0 00000100:00100000:3.0:1789760735.587806:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5756 00000100:00100000:3.0:1789760735.587820:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745662796773 time 745773128357 state 0x2 ref 0 00000100:00100000:0.0:1789760735.591295:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760735.591402:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493714944 00000100:00100000:2.0:1789760735.591414:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493714944 00000100:00100000:2.0:1789760735.591489:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745662796773 time 745773128357 state 0x0 ref 1 00000100:00100000:2.0:1789760735.591508:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745773388773 time 745773128357 state 0x1 ref 1 00000100:00100000:2.0:1789760735.591552:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745773388773 time 745932586854 state 0x0 ref 1 00000100:00100000:2.0:1789760735.591567:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493714944:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760735.591642:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23457792, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xb807 00010000:00010000:2.0:1789760735.591653:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e052c3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.591671:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760735.591676:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.591683:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e052c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.591699:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e052c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760735.647174:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973046 is committed 00002000:00010000:1.0:1789760735.647335:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23457792 -> 23461888 00002000:00010000:1.0:1789760735.647341:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760734 -> 1789760735 00002000:00010000:1.0:1789760735.647344:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760734 -> 1789760735 00010000:00010000:1.0:1789760735.647363:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e052c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.647378:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e052c3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.647389:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e052c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.647398:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760735.647403:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e052c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.647414:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e052c3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.647441:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e052c3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.647449:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e052c3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.647767:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493714944:12345-192.168.203.60@tcp:4:dd.0 Request processed in 56194us (56477us total) trans 4294973046 rc 0/0 00000100:00100000:1.0:1789760735.647780:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5757 00000100:00100000:1.0:1789760735.647793:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745773388773 time 745932586854 state 0x2 ref 0 00000100:00100000:1.0:1789760735.654387:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760735.654510:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493715072 00000100:00100000:0.0:1789760735.654534:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493715072 00000100:00100000:0.0:1789760735.654619:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745773388773 time 745932586854 state 0x0 ref 1 00000100:00100000:0.0:1789760735.654640:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745932847270 time 745932586854 state 0x1 ref 1 00000100:00100000:0.0:1789760735.654678:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745932847270 time 745995713542 state 0x0 ref 1 00000100:00100000:0.0:1789760735.654694:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493715072:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760735.654765:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23461888, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb807 00010000:00010000:0.0:1789760735.654786:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e052ca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.654806:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=745s v=(0 0) 00010000:00010000:0.0:1789760735.654811:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760735.654829:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e052ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.655023:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e052ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760735.710411:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973047 is committed 00002000:00010000:1.0:1789760735.710606:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23461888 -> 23465984 00010000:00010000:1.0:1789760735.710628:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e052ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.710649:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e052ca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.710660:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e052ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.710667:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760735.710673:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e052ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.710684:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e052ca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.710714:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e052ca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.710723:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e052ca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.710854:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493715072:12345-192.168.203.60@tcp:4:dd.0 Request processed in 56154us (56475us total) trans 4294973047 rc 0/0 00000100:00100000:1.0:1789760735.710867:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5758 00000100:00100000:1.0:1789760735.710881:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745932847270 time 745995713542 state 0x2 ref 0 00000100:00100000:2.0:1789760735.717685:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760735.717793:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493715200 00000100:00100000:3.0:1789760735.717807:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493715200 00000100:00100000:3.0:1789760735.717877:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745932847270 time 745995713542 state 0x0 ref 1 00000100:00100000:3.0:1789760735.717896:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745995973958 time 745995713542 state 0x1 ref 1 00000100:00100000:3.0:1789760735.717932:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745995973958 time 746058967418 state 0x0 ref 1 00000100:00100000:3.0:1789760735.717947:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493715200:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760735.718039:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23465984, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb807 00010000:00010000:3.0:1789760735.718051:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052d1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.718073:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:3.0:1789760735.718079:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.718086:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.718102:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760735.755862:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973048 is committed 00002000:00010000:0.0:1789760735.756048:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23465984 -> 23470080 00010000:00010000:0.0:1789760735.756068:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.756084:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052d1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.756094:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.756102:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760735.756108:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.756120:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052d1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.756145:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052d1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.756155:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e052d1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760735.756280:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493715200:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38329us (38603us total) trans 4294973048 rc 0/0 00000100:00100000:0.0:1789760735.756292:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5759 00000100:00100000:0.0:1789760735.756306:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745995973958 time 746058967418 state 0x2 ref 0 00000100:00100000:1.0:1789760735.760188:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760735.760302:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493715328 00000100:00100000:3.0:1789760735.760315:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493715328 00000100:00100000:3.0:1789760735.760394:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 745995973958 time 746058967418 state 0x0 ref 1 00000100:00100000:3.0:1789760735.760413:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746059227834 time 746058967418 state 0x1 ref 1 00000100:00100000:3.0:1789760735.760683:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746059227834 time 746101716026 state 0x0 ref 1 00000100:00100000:3.0:1789760735.760701:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493715328:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760735.760773:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23470080, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb807 00010000:00010000:3.0:1789760735.760785:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e052d8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.760805:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:3.0:1789760735.760810:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.760818:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e052d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.760841:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e052d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760735.792914:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973049 is committed 00002000:00010000:1.0:1789760735.793253:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23470080 -> 23474176 00010000:00010000:1.0:1789760735.793272:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e052d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.793287:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e052d8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.793297:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e052d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.793305:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760735.793311:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e052d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.793321:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e052d8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.793346:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e052d8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.793354:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e052d8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.793492:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493715328:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32789us (33312us total) trans 4294973049 rc 0/0 00000100:00100000:1.0:1789760735.793505:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5760 00000100:00100000:1.0:1789760735.793518:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746059227834 time 746101716026 state 0x2 ref 0 00000100:00100000:1.0:1789760735.801275:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760735.801661:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493715456 00000100:00100000:2.0:1789760735.801675:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493715456 00000100:00100000:2.0:1789760735.801763:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746059227834 time 746101716026 state 0x0 ref 1 00000100:00100000:2.0:1789760735.801781:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746101976442 time 746101716026 state 0x1 ref 1 00000100:00100000:2.0:1789760735.801816:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746101976442 time 746142851083 state 0x0 ref 1 00000100:00100000:2.0:1789760735.801830:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493715456:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760735.801893:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23474176, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb807 00010000:00010000:2.0:1789760735.801904:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e052df lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.801924:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760735.801930:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.801938:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e052df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.801955:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e052df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.802160:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760735.847027:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23474176 -> 23478272 00010000:00010000:2.0:1789760735.847054:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e052df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.847070:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e052df lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.847081:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e052df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.847088:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760735.847095:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e052df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.847105:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e052df lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.847130:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e052df lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.847138:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e052df lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.847232:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973050 is committed 00000100:00100000:2.0:1789760735.847269:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493715456:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45434us (46002us total) trans 4294973050 rc 0/0 00000100:00100000:2.0:1789760735.847286:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5761 00000100:00100000:2.0:1789760735.847298:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746101976442 time 746142851083 state 0x2 ref 0 00000100:00100000:0.0:1789760735.850487:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760735.850838:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493715584 00000100:00100000:1.0:1789760735.850853:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493715584 00000100:00100000:1.0:1789760735.850938:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746101976442 time 746142851083 state 0x0 ref 1 00000100:00100000:1.0:1789760735.851218:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746143111499 time 746142851083 state 0x1 ref 1 00000100:00100000:1.0:1789760735.851327:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746143111499 time 746192361007 state 0x0 ref 1 00000100:00100000:1.0:1789760735.851344:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493715584:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760735.851422:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23478272, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb807 00010000:00010000:1.0:1789760735.851435:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e052e6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.851458:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760735.851463:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.851472:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e052e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.851490:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e052e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760735.884815:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973051 is committed 00002000:00010000:1.0:1789760735.885191:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23478272 -> 23482368 00010000:00010000:1.0:1789760735.885212:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e052e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.885233:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e052e6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.885244:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e052e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.885252:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760735.885258:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e052e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.885269:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e052e6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.885296:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e052e6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.885305:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e052e6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.885478:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493715584:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34129us (34996us total) trans 4294973051 rc 0/0 00000100:00100000:1.0:1789760735.885491:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5762 00000100:00100000:1.0:1789760735.885504:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746143111499 time 746192361007 state 0x2 ref 0 00000100:00100000:3.0:1789760735.890653:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760735.890735:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493715712 00000100:00100000:1.0:1789760735.890750:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493715712 00000100:00100000:1.0:1789760735.890830:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746143111499 time 746192361007 state 0x0 ref 1 00000100:00100000:1.0:1789760735.890850:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746192621423 time 746192361007 state 0x1 ref 1 00000100:00100000:1.0:1789760735.890887:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746192621423 time 746231922190 state 0x0 ref 1 00000100:00100000:1.0:1789760735.890902:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493715712:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760735.891011:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23482368, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb807 00010000:00010000:1.0:1789760735.891022:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e052ed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.891042:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760735.891047:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.891055:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e052ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.891073:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e052ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760735.939547:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23482368 -> 23486464 00010000:00010000:0.0:1789760735.939579:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e052ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.939597:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e052ed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760735.939604:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973052 is committed 00010000:00010000:0.0:1789760735.939606:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e052ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.939615:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760735.939620:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e052ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.939630:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e052ed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.939655:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e052ed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.939663:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e052ed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760735.939801:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493715712:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48893us (49158us total) trans 4294973052 rc 0/0 00000100:00100000:0.0:1789760735.939815:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5763 00000100:00100000:0.0:1789760735.939829:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746192621423 time 746231922190 state 0x2 ref 0 00000100:00100000:0.0:1789760735.943436:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760735.943593:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493715840 00000100:00100000:3.0:1789760735.943606:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493715840 00000100:00100000:3.0:1789760735.943675:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746192621423 time 746231922190 state 0x0 ref 1 00000100:00100000:3.0:1789760735.943694:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746232182606 time 746231922190 state 0x1 ref 1 00000100:00100000:3.0:1789760735.943727:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746232182606 time 746284762321 state 0x0 ref 1 00000100:00100000:3.0:1789760735.943740:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493715840:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760735.943805:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23486464, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb807 00010000:00010000:3.0:1789760735.943817:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e052f4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.943836:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:3.0:1789760735.943840:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.943848:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e052f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.943864:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e052f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760735.988061:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23486464 -> 23490560 00010000:00010000:3.0:1789760735.988143:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e052f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.988159:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e052f4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.988169:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e052f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.988177:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760735.988182:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e052f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.988192:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e052f4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.988220:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e052f4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.988227:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e052f4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.988392:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493715840:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44644us (44961us total) trans 4294973053 rc 0/0 00000100:00100000:3.0:1789760735.988410:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5764 00000100:00100000:3.0:1789760735.988423:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746232182606 time 746284762321 state 0x2 ref 0 00000001:00080000:1.0:1789760735.988511:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973053 is committed 00000100:00100000:2.0:1789760735.991846:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760735.992058:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493715968 00000100:00100000:1.0:1789760735.992071:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493715968 00000100:00100000:1.0:1789760735.992199:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746232182606 time 746284762321 state 0x0 ref 1 00000100:00100000:1.0:1789760735.992218:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746285022737 time 746284762321 state 0x1 ref 1 00000100:00100000:1.0:1789760735.992295:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746285022737 time 746333328990 state 0x0 ref 1 00000100:00100000:1.0:1789760735.992311:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493715968:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760735.992378:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23490560, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb807 00010000:00010000:1.0:1789760735.992388:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e052fb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.992408:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760735.992414:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.992421:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e052fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.992439:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e052fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.037529:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973054 is committed 00002000:00010000:1.0:1789760736.037936:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23490560 -> 23494656 00010000:00010000:1.0:1789760736.037957:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e052fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.038016:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e052fb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.038028:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e052fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.038036:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760736.038042:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e052fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.038053:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e052fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.038193:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e052fb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.038204:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e052fb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.038318:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493715968:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46002us (46479us total) trans 4294973054 rc 0/0 00000100:00100000:1.0:1789760736.038331:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5765 00000100:00100000:1.0:1789760736.038344:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746285022737 time 746333328990 state 0x2 ref 0 00000100:00100000:0.0:1789760736.041351:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760736.041728:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493716096 00000100:00100000:1.0:1789760736.041742:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493716096 00000100:00100000:1.0:1789760736.041824:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746285022737 time 746333328990 state 0x0 ref 1 00000100:00100000:1.0:1789760736.041843:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746333589406 time 746333328990 state 0x1 ref 1 00000100:00100000:1.0:1789760736.042163:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746333589406 time 746383196602 state 0x0 ref 1 00000100:00100000:1.0:1789760736.042180:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493716096:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760736.042251:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23494656, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb807 00010000:00010000:1.0:1789760736.042261:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05302 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.042282:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760736.042287:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.042294:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05302 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.042310:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05302 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760736.076828:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973055 is committed 00002000:00010000:1.0:1789760736.076989:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23494656 -> 23498752 00010000:00010000:1.0:1789760736.077175:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05302 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.077191:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05302 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.077202:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05302 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.077211:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760736.077217:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05302 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.077228:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05302 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.077255:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05302 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.077265:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05302 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.077431:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493716096:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35248us (36087us total) trans 4294973055 rc 0/0 00000100:00100000:1.0:1789760736.077443:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5766 00000100:00100000:1.0:1789760736.077455:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746333589406 time 746383196602 state 0x2 ref 0 00000100:00100000:0.0:1789760736.082537:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760736.082699:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493716224 00000100:00100000:1.0:1789760736.082712:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493716224 00000100:00100000:1.0:1789760736.082788:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746333589406 time 746383196602 state 0x0 ref 1 00000100:00100000:1.0:1789760736.082807:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746383457018 time 746383196602 state 0x1 ref 1 00000100:00100000:1.0:1789760736.082910:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746383457018 time 746423943388 state 0x0 ref 1 00000100:00100000:1.0:1789760736.082926:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493716224:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760736.083084:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23498752, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb807 00010000:00010000:1.0:1789760736.083095:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05309 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.083114:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760736.083119:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.083126:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05309 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.083147:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05309 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760736.114947:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23498752 -> 23502848 00010000:00010000:0.0:1789760736.115009:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05309 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.115032:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05309 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.115044:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05309 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.115053:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760736.115059:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05309 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.115070:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05309 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.115095:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05309 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.115104:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05309 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.115224:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493716224:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32294us (32705us total) trans 4294973056 rc 0/0 00000100:00100000:0.0:1789760736.115236:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5767 00000100:00100000:0.0:1789760736.115249:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746383457018 time 746423943388 state 0x2 ref 0 00000001:00080000:2.0:1789760736.116124:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973056 is committed 00000100:00100000:2.0:1789760736.118870:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760736.119087:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493716352 00000100:00100000:3.0:1789760736.119100:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493716352 00000100:00100000:3.0:1789760736.119175:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746383457018 time 746423943388 state 0x0 ref 1 00000100:00100000:3.0:1789760736.119195:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746424203804 time 746423943388 state 0x1 ref 1 00000100:00100000:3.0:1789760736.119281:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746424203804 time 746460315113 state 0x0 ref 1 00000100:00100000:3.0:1789760736.119297:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493716352:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760736.119365:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23502848, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb807 00010000:00010000:3.0:1789760736.119377:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05310 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.119398:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:3.0:1789760736.119403:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.119411:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05310 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.119430:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05310 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760736.157370:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23502848 -> 23506944 00000001:00080000:2.0:1789760736.157381:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973057 is committed 00010000:00010000:3.0:1789760736.157401:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05310 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.157418:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05310 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.157428:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05310 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.157437:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760736.157443:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05310 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.157454:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05310 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.157480:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05310 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.157489:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05310 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.157727:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493716352:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38426us (38866us total) trans 4294973057 rc 0/0 00000100:00100000:3.0:1789760736.157741:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5768 00000100:00100000:3.0:1789760736.157754:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746424203804 time 746460315113 state 0x2 ref 0 00000100:00100000:1.0:1789760736.161676:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760736.163717:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493716480 00000100:00100000:2.0:1789760736.163737:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493716480 00000100:00100000:2.0:1789760736.163826:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746424203804 time 746460315113 state 0x0 ref 1 00000100:00100000:2.0:1789760736.163847:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746460575529 time 746460315113 state 0x1 ref 1 00000100:00100000:2.0:1789760736.163883:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746460575529 time 746504918400 state 0x0 ref 1 00000100:00100000:2.0:1789760736.163900:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493716480:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760736.163982:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23506944, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb807 00010000:00010000:2.0:1789760736.166820:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05317 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.166863:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760736.166869:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.166878:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05317 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.166904:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05317 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.239515:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973058 is committed 00002000:00010000:1.0:1789760736.244801:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23506944 -> 23511040 00010000:00010000:1.0:1789760736.244831:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05317 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.244845:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05317 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.244855:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05317 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.244863:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760736.244868:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05317 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.244878:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05317 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.244903:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05317 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.244910:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05317 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.245088:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493716480:12345-192.168.203.60@tcp:4:dd.0 Request processed in 81184us (83419us total) trans 4294973058 rc 0/0 00000100:00100000:1.0:1789760736.245101:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5769 00000100:00100000:1.0:1789760736.245113:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746460575529 time 746504918400 state 0x2 ref 0 00000100:00100000:0.0:1789760736.250416:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760736.251747:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493716608 00000100:00100000:1.0:1789760736.251765:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493716608 00000100:00100000:1.0:1789760736.251853:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746460575529 time 746504918400 state 0x0 ref 1 00000100:00100000:1.0:1789760736.251873:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746505178816 time 746504918400 state 0x1 ref 1 00000100:00100000:1.0:1789760736.251910:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746505178816 time 746592944924 state 0x0 ref 1 00000100:00100000:1.0:1789760736.251926:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493716608:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760736.252022:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23511040, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb807 00010000:00010000:1.0:1789760736.252034:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0531e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.252055:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760736.252060:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.252068:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0531e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.252085:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0531e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.319925:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973059 is committed 00002000:00010000:1.0:1789760736.320187:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23511040 -> 23515136 00010000:00010000:1.0:1789760736.320208:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0531e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.320223:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0531e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.320233:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0531e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.320241:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760736.320246:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0531e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.320257:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0531e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.320284:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0531e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.320293:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0531e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.320415:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493716608:12345-192.168.203.60@tcp:4:dd.0 Request processed in 68485us (70006us total) trans 4294973059 rc 0/0 00000100:00100000:1.0:1789760736.320432:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5770 00000100:00100000:1.0:1789760736.320445:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746505178816 time 746592944924 state 0x2 ref 0 00000100:00100000:0.0:1789760736.324581:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760736.324679:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493716736 00000100:00100000:1.0:1789760736.324691:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493716736 00000100:00100000:1.0:1789760736.324763:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746505178816 time 746592944924 state 0x0 ref 1 00000100:00100000:1.0:1789760736.324793:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746593205340 time 746592944924 state 0x1 ref 1 00000100:00100000:1.0:1789760736.324879:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746593205340 time 746665914044 state 0x0 ref 1 00000100:00100000:1.0:1789760736.324894:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493716736:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760736.324995:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23515136, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb807 00010000:00010000:1.0:1789760736.325006:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05325 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.325025:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760736.325030:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.325038:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05325 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.325055:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05325 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.371461:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973060 is committed 00002000:00010000:1.0:1789760736.371752:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23515136 -> 23519232 00010000:00010000:1.0:1789760736.371772:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05325 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.371787:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05325 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.371797:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05325 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.371805:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760736.371811:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05325 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.371821:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05325 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.371846:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05325 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.371853:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05325 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.372056:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493716736:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47157us (47491us total) trans 4294973060 rc 0/0 00000100:00100000:1.0:1789760736.372069:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5771 00000100:00100000:1.0:1789760736.372081:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746593205340 time 746665914044 state 0x2 ref 0 00000100:00100000:0.0:1789760736.375137:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760736.375504:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493716864 00000100:00100000:3.0:1789760736.375527:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493716864 00000100:00100000:3.0:1789760736.375658:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746593205340 time 746665914044 state 0x0 ref 1 00000100:00100000:3.0:1789760736.375677:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746666174460 time 746665914044 state 0x1 ref 1 00000100:00100000:3.0:1789760736.375752:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746666174460 time 746716786002 state 0x0 ref 1 00000100:00100000:3.0:1789760736.375768:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493716864:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760736.375861:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23519232, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb807 00010000:00010000:3.0:1789760736.375872:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0532c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.375890:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:3.0:1789760736.375895:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.375903:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0532c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.375919:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0532c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.406762:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973061 is committed 00002000:00010000:2.0:1789760736.407235:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23519232 -> 23523328 00010000:00010000:2.0:1789760736.407255:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0532c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.407271:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0532c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.407280:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0532c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.407288:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760736.407294:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0532c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.407304:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0532c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.407330:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0532c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.407338:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0532c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.407467:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493716864:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31696us (32338us total) trans 4294973061 rc 0/0 00000100:00100000:2.0:1789760736.407480:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5772 00000100:00100000:2.0:1789760736.407493:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746666174460 time 746716786002 state 0x2 ref 0 00000100:00100000:0.0:1789760736.411427:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760736.411538:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493716992 00000100:00100000:2.0:1789760736.411551:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493716992 00000100:00100000:2.0:1789760736.411631:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746666174460 time 746716786002 state 0x0 ref 1 00000100:00100000:2.0:1789760736.411650:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746717046418 time 746716786002 state 0x1 ref 1 00000100:00100000:2.0:1789760736.411850:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746717046418 time 746752883830 state 0x0 ref 1 00000100:00100000:2.0:1789760736.411867:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493716992:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760736.411934:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23523328, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb807 00010000:00010000:2.0:1789760736.411946:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05333 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.411997:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760736.412003:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.412011:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05333 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.412028:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05333 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760736.443174:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23523328 -> 23527424 00010000:00010000:3.0:1789760736.443203:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05333 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.443225:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05333 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.443235:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05333 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.443244:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760736.443251:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05333 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.443262:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05333 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.443287:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05333 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.443296:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05333 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.443428:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493716992:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31557us (32009us total) trans 4294973062 rc 0/0 00000100:00100000:3.0:1789760736.443441:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5773 00000100:00100000:3.0:1789760736.443454:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746717046418 time 746752883830 state 0x2 ref 0 00000001:00080000:0.0:1789760736.443558:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973062 is committed 00000100:00100000:0.0:1789760736.447273:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760736.447427:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493717120 00000100:00100000:1.0:1789760736.447443:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493717120 00000100:00100000:1.0:1789760736.447533:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746717046418 time 746752883830 state 0x0 ref 1 00000100:00100000:1.0:1789760736.447552:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746753144246 time 746752883830 state 0x1 ref 1 00000100:00100000:1.0:1789760736.447639:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746753144246 time 746788673780 state 0x0 ref 1 00000100:00100000:1.0:1789760736.447655:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493717120:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760736.447723:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23527424, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb807 00010000:00010000:1.0:1789760736.447734:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0533a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.447755:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760736.447760:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.447779:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0533a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.447799:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0533a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760736.493009:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973063 is committed 00002000:00010000:1.0:1789760736.493187:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23527424 -> 23531520 00010000:00010000:1.0:1789760736.493207:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0533a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.493223:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0533a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.493239:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0533a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.493247:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760736.493253:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0533a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.493263:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0533a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.493288:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0533a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.493297:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0533a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.493432:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493717120:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45773us (46167us total) trans 4294973063 rc 0/0 00000100:00100000:1.0:1789760736.493445:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5774 00000100:00100000:1.0:1789760736.493457:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746753144246 time 746788673780 state 0x2 ref 0 00000100:00100000:3.0:1789760736.500215:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760736.500341:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493717248 00000100:00100000:1.0:1789760736.500354:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493717248 00000100:00100000:1.0:1789760736.500432:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746753144246 time 746788673780 state 0x0 ref 1 00000100:00100000:1.0:1789760736.500452:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746788934196 time 746788673780 state 0x1 ref 1 00000100:00100000:1.0:1789760736.500537:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746788934196 time 746841570914 state 0x0 ref 1 00000100:00100000:1.0:1789760736.500552:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493717248:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760736.500616:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23531520, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb807 00010000:00010000:1.0:1789760736.500628:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05341 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.500649:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760736.500654:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.500663:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05341 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.500683:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05341 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.584433:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973064 is committed 00002000:00010000:2.0:1789760736.584588:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23531520 -> 23535616 00010000:00010000:2.0:1789760736.584610:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05341 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.584628:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05341 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.584638:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05341 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.584647:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760736.584652:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05341 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.584663:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05341 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.584690:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05341 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.584699:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05341 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.584906:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493717248:12345-192.168.203.60@tcp:4:dd.0 Request processed in 84350us (84706us total) trans 4294973064 rc 0/0 00000100:00100000:2.0:1789760736.584921:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5775 00000100:00100000:2.0:1789760736.584934:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746788934196 time 746841570914 state 0x2 ref 0 00000100:00100000:0.0:1789760736.591137:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760736.591365:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493717376 00000100:00100000:2.0:1789760736.591380:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493717376 00000100:00100000:2.0:1789760736.591457:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746788934196 time 746841570914 state 0x0 ref 1 00000100:00100000:2.0:1789760736.591476:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746841831330 time 746841570914 state 0x1 ref 1 00000100:00100000:2.0:1789760736.591512:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746841831330 time 746932546347 state 0x0 ref 1 00000100:00100000:2.0:1789760736.591536:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493717376:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760736.591602:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23535616, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xb807 00010000:00010000:2.0:1789760736.591614:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05348 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.591633:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760736.591639:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.591647:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05348 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.591663:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05348 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760736.650657:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973065 is committed 00002000:00010000:3.0:1789760736.650853:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23535616 -> 23539712 00002000:00010000:3.0:1789760736.650858:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760735 -> 1789760736 00002000:00010000:3.0:1789760736.650929:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760735 -> 1789760736 00010000:00010000:3.0:1789760736.650946:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05348 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.650983:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05348 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.651002:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05348 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.651010:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760736.651016:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05348 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.651027:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05348 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.651052:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05348 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.651060:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05348 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.651184:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493717376:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59644us (60054us total) trans 4294973065 rc 0/0 00000100:00100000:3.0:1789760736.651197:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5776 00000100:00100000:3.0:1789760736.651211:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746841831330 time 746932546347 state 0x2 ref 0 00000100:00100000:0.0:1789760736.653934:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760736.654229:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493717504 00000100:00100000:3.0:1789760736.654242:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493717504 00000100:00100000:3.0:1789760736.654314:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746841831330 time 746932546347 state 0x0 ref 1 00000100:00100000:3.0:1789760736.654333:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746932806763 time 746932546347 state 0x1 ref 1 00000100:00100000:3.0:1789760736.654370:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746932806763 time 746995405658 state 0x0 ref 1 00000100:00100000:3.0:1789760736.654384:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493717504:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760736.654452:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23539712, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb807 00010000:00010000:3.0:1789760736.654463:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0534f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.654483:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=746s v=(0 0) 00010000:00010000:3.0:1789760736.654489:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.654496:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0534f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.654524:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0534f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760736.712645:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973066 is committed 00002000:00010000:3.0:1789760736.713022:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23539712 -> 23543808 00010000:00010000:3.0:1789760736.713044:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0534f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.713060:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0534f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.713071:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0534f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.713079:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760736.713085:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0534f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.713095:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0534f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.713124:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0534f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.713133:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0534f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.713259:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493717504:12345-192.168.203.60@tcp:4:dd.0 Request processed in 58869us (59403us total) trans 4294973066 rc 0/0 00000100:00100000:3.0:1789760736.713273:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5777 00000100:00100000:3.0:1789760736.713287:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746932806763 time 746995405658 state 0x2 ref 0 00000100:00100000:0.0:1789760736.724064:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760736.724157:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493717632 00000100:00100000:2.0:1789760736.724171:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493717632 00000100:00100000:2.0:1789760736.724243:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746932806763 time 746995405658 state 0x0 ref 1 00000100:00100000:2.0:1789760736.724262:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746995666074 time 746995405658 state 0x1 ref 1 00000100:00100000:2.0:1789760736.724297:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746995666074 time 747065331656 state 0x0 ref 1 00000100:00100000:2.0:1789760736.724314:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493717632:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760736.724381:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23543808, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb807 00010000:00010000:2.0:1789760736.724392:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05356 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.724413:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760736.724419:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.724427:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05356 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.724443:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05356 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.773930:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973067 is committed 00002000:00010000:1.0:1789760736.774252:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23543808 -> 23547904 00010000:00010000:1.0:1789760736.774278:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05356 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.774296:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05356 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.774307:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05356 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.774321:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760736.774327:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05356 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.774338:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05356 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.774365:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05356 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.774373:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05356 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.774571:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493717632:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50249us (50508us total) trans 4294973067 rc 0/0 00000100:00100000:1.0:1789760736.774585:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5778 00000100:00100000:1.0:1789760736.774597:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746995666074 time 747065331656 state 0x2 ref 0 00000100:00100000:0.0:1789760736.783762:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760736.783896:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493717760 00000100:00100000:2.0:1789760736.783911:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493717760 00000100:00100000:2.0:1789760736.784250:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 746995666074 time 747065331656 state 0x0 ref 1 00000100:00100000:2.0:1789760736.784271:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747065592072 time 747065331656 state 0x1 ref 1 00000100:00100000:2.0:1789760736.784309:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747065592072 time 747125343956 state 0x0 ref 1 00000100:00100000:2.0:1789760736.784325:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493717760:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760736.784393:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23547904, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb807 00010000:00010000:2.0:1789760736.784403:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0535d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.784423:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760736.784429:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.784437:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0535d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.784455:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0535d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.828875:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760736.842369:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973068 is committed 00002000:00010000:2.0:1789760736.842485:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23547904 -> 23552000 00010000:00010000:2.0:1789760736.842506:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0535d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.842530:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0535d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.842540:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0535d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.842547:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760736.842553:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0535d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.842563:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0535d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.842653:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0535d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.842662:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0535d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.842848:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493717760:12345-192.168.203.60@tcp:4:dd.0 Request processed in 58520us (59096us total) trans 4294973068 rc 0/0 00000100:00100000:2.0:1789760736.842862:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5779 00000100:00100000:2.0:1789760736.842873:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747065592072 time 747125343956 state 0x2 ref 0 00000100:00100000:1.0:1789760736.847919:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760736.848195:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493717888 00000100:00100000:3.0:1789760736.848208:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493717888 00000100:00100000:3.0:1789760736.848288:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747065592072 time 747125343956 state 0x0 ref 1 00000100:00100000:3.0:1789760736.848307:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747125604372 time 747125343956 state 0x1 ref 1 00000100:00100000:3.0:1789760736.848376:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747125604372 time 747189410724 state 0x0 ref 1 00000100:00100000:3.0:1789760736.848391:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493717888:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760736.848460:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23552000, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb807 00010000:00010000:3.0:1789760736.848471:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05364 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.848492:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:3.0:1789760736.848498:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.848506:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05364 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.848534:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05364 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760736.907427:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973069 is committed 00002000:00010000:3.0:1789760736.907687:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23552000 -> 23556096 00010000:00010000:3.0:1789760736.907708:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05364 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.907725:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05364 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.907735:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05364 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.907744:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760736.907750:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05364 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.907760:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05364 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.907788:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05364 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.907797:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05364 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.907937:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493717888:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59540us (60027us total) trans 4294973069 rc 0/0 00000100:00100000:3.0:1789760736.907949:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5780 00000100:00100000:3.0:1789760736.907998:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747125604372 time 747189410724 state 0x2 ref 0 00000100:00100000:0.0:1789760736.916138:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760736.916209:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493718016 00000100:00100000:3.0:1789760736.916223:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493718016 00000100:00100000:3.0:1789760736.916310:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747125604372 time 747189410724 state 0x0 ref 1 00000100:00100000:3.0:1789760736.916329:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747189671140 time 747189410724 state 0x1 ref 1 00000100:00100000:3.0:1789760736.916365:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747189671140 time 747257401176 state 0x0 ref 1 00000100:00100000:3.0:1789760736.916380:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493718016:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760736.916459:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23556096, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb807 00010000:00010000:3.0:1789760736.916470:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0536b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.916489:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:3.0:1789760736.916495:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.916503:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0536b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.916531:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0536b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760736.971647:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973070 is committed 00002000:00010000:0.0:1789760736.972094:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23556096 -> 23560192 00010000:00010000:0.0:1789760736.972115:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0536b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.972131:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0536b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.972141:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0536b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.972150:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760736.972156:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0536b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.972167:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0536b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.972200:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0536b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.972209:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0536b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.972332:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493718016:12345-192.168.203.60@tcp:4:dd.0 Request processed in 55947us (56202us total) trans 4294973070 rc 0/0 00000100:00100000:0.0:1789760736.972345:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5781 00000100:00100000:0.0:1789760736.972358:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747189671140 time 747257401176 state 0x2 ref 0 00000100:00100000:1.0:1789760736.975490:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760736.975653:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493718144 00000100:00100000:2.0:1789760736.975667:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493718144 00000100:00100000:2.0:1789760736.975759:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747189671140 time 747257401176 state 0x0 ref 1 00000100:00100000:2.0:1789760736.975779:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747257661592 time 747257401176 state 0x1 ref 1 00000100:00100000:2.0:1789760736.975814:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747257661592 time 747316848852 state 0x0 ref 1 00000100:00100000:2.0:1789760736.975829:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493718144:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760736.975900:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23560192, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb807 00010000:00010000:2.0:1789760736.975912:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05372 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.975932:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760736.975938:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.975947:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05372 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.976006:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05372 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760737.020632:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23560192 -> 23564288 00010000:00010000:2.0:1789760737.020662:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05372 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.020679:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05372 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.020689:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05372 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.020698:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760737.020822:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05372 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.020835:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05372 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.020860:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05372 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.020869:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05372 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.021113:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493718144:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45278us (45629us total) trans 4294973071 rc 0/0 00000100:00100000:2.0:1789760737.021128:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5782 00000100:00100000:2.0:1789760737.021141:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747257661592 time 747316848852 state 0x2 ref 0 00000001:00080000:1.0:1789760737.022007:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973071 is committed 00000100:00100000:1.0:1789760737.023563:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760737.023643:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493718272 00000100:00100000:2.0:1789760737.023655:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493718272 00000100:00100000:2.0:1789760737.023760:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747257661592 time 747316848852 state 0x0 ref 1 00000100:00100000:2.0:1789760737.023781:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747317109268 time 747316848852 state 0x1 ref 1 00000100:00100000:2.0:1789760737.023818:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747317109268 time 747364853046 state 0x0 ref 1 00000100:00100000:2.0:1789760737.023834:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb05450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493718272:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760737.023896:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23564288, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb807 00010000:00010000:2.0:1789760737.023907:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05379 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.023926:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760737.023931:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.023939:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05379 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.023956:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05379 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760737.024770:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789760737.070219:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23564288 -> 23568384 00010000:00010000:2.0:1789760737.070247:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05379 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.070261:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05379 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.070270:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05379 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.070277:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760737.070282:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05379 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.070291:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05379 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.070315:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05379 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.070322:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05379 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.070447:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb05450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493718272:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46609us (46888us total) trans 4294973072 rc 0/0 00000100:00100000:2.0:1789760737.070462:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5783 00000100:00100000:2.0:1789760737.070474:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747317109268 time 747364853046 state 0x2 ref 0 00000001:00080000:1.0:1789760737.070934:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973072 is committed 00000100:00100000:2.0:1789760737.073924:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760737.074122:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493718400 00000100:00100000:3.0:1789760737.074134:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493718400 00000100:00100000:3.0:1789760737.074219:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747317109268 time 747364853046 state 0x0 ref 1 00000100:00100000:3.0:1789760737.074238:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747365113462 time 747364853046 state 0x1 ref 1 00000100:00100000:3.0:1789760737.074272:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747365113462 time 747415307751 state 0x0 ref 1 00000100:00100000:3.0:1789760737.074286:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493718400:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760737.074348:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23568384, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb807 00010000:00010000:3.0:1789760737.074359:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05380 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.074377:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:3.0:1789760737.074382:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.074390:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05380 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.074406:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05380 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760737.120632:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23568384 -> 23572480 00010000:00010000:0.0:1789760737.120662:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05380 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.120679:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05380 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.120689:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05380 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.120698:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760737.120704:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05380 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.120721:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05380 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.120746:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05380 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.120754:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05380 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760737.121019:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493718400:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46670us (47043us total) trans 4294973073 rc 0/0 00000100:00100000:0.0:1789760737.121033:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5784 00000100:00100000:0.0:1789760737.121046:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747365113462 time 747415307751 state 0x2 ref 0 00000001:00080000:2.0:1789760737.121195:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973073 is committed 00000100:00100000:0.0:1789760737.127344:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760737.127443:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493718528 00000100:00100000:1.0:1789760737.127457:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493718528 00000100:00100000:1.0:1789760737.127540:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747365113462 time 747415307751 state 0x0 ref 1 00000100:00100000:1.0:1789760737.127561:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747415568167 time 747415307751 state 0x1 ref 1 00000100:00100000:1.0:1789760737.127608:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747415568167 time 747468642178 state 0x0 ref 1 00000100:00100000:1.0:1789760737.127625:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493718528:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760737.127693:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23572480, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb807 00010000:00010000:1.0:1789760737.127704:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05387 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.127724:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760737.127729:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760737.127737:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05387 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.127754:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05387 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760737.168283:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23572480 -> 23576576 00010000:00010000:1.0:1789760737.168311:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05387 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.168327:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05387 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.168338:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05387 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.168346:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760737.168352:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05387 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.168362:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05387 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.168388:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05387 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.168397:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05387 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760737.168535:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493718528:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40906us (41199us total) trans 4294973074 rc 0/0 00000100:00100000:1.0:1789760737.168548:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5785 00000100:00100000:1.0:1789760737.168560:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747415568167 time 747468642178 state 0x2 ref 0 00000100:00100000:1.0:1789760737.171441:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760737.171552:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493718656 00000100:00100000:2.0:1789760737.171564:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493718656 00000100:00100000:2.0:1789760737.171649:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747415568167 time 747468642178 state 0x0 ref 1 00000100:00100000:2.0:1789760737.171669:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747468902594 time 747468642178 state 0x1 ref 1 00000100:00100000:2.0:1789760737.171710:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747468902594 time 747512743992 state 0x0 ref 1 00000100:00100000:2.0:1789760737.171725:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493718656:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760737.171792:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23576576, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb807 00010000:00010000:2.0:1789760737.171803:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0538e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.171823:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760737.171829:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.171837:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0538e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.171852:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0538e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760737.175491:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973074 is committed 00002000:00010000:1.0:1789760737.208866:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23576576 -> 23580672 00010000:00010000:1.0:1789760737.208896:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0538e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.208919:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0538e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.208930:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0538e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.208938:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760737.208944:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0538e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.208954:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0538e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.209115:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0538e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.209125:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0538e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760737.209255:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493718656:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37525us (37821us total) trans 4294973075 rc 0/0 00000100:00100000:1.0:1789760737.209267:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5786 00000100:00100000:1.0:1789760737.209279:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747468902594 time 747512743992 state 0x2 ref 0 00000100:00100000:3.0:1789760737.212765:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:0.0:1789760737.213433:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973075 is committed 00000100:00100000:1.0:1789760737.213592:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493718784 00000100:00100000:1.0:1789760737.213606:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493718784 00000100:00100000:1.0:1789760737.213685:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747468902594 time 747512743992 state 0x0 ref 1 00000100:00100000:1.0:1789760737.213704:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747513004408 time 747512743992 state 0x1 ref 1 00000100:00100000:1.0:1789760737.213738:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747513004408 time 747554772934 state 0x0 ref 1 00000100:00100000:1.0:1789760737.213754:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493718784:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760737.213822:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23580672, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb807 00010000:00010000:1.0:1789760737.213833:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05395 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.213855:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760737.213861:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760737.213869:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05395 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.213886:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05395 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760737.247556:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23580672 -> 23584768 00010000:00010000:1.0:1789760737.247595:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05395 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.247612:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05395 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.247628:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05395 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.247637:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760737.247643:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05395 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.247654:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05395 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.247680:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05395 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.247689:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05395 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760737.247830:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493718784:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34071us (35072us total) trans 4294973076 rc 0/0 00000100:00100000:1.0:1789760737.247843:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5787 00000100:00100000:1.0:1789760737.247857:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747513004408 time 747554772934 state 0x2 ref 0 00000001:00080000:3.0:1789760737.249296:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973076 is committed 00000100:00100000:3.0:1789760737.251544:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760737.251702:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493718912 00000100:00100000:0.0:1789760737.251717:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493718912 00000100:00100000:0.0:1789760737.251799:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747513004408 time 747554772934 state 0x0 ref 1 00000100:00100000:0.0:1789760737.251818:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747555033350 time 747554772934 state 0x1 ref 1 00000100:00100000:0.0:1789760737.251856:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747555033350 time 747592891050 state 0x0 ref 1 00000100:00100000:0.0:1789760737.251870:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493718912:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760737.251936:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23584768, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb807 00010000:00010000:0.0:1789760737.251948:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0539c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.252014:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:0.0:1789760737.252020:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760737.252028:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0539c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.252047:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0539c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760737.303594:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973077 is committed 00002000:00010000:3.0:1789760737.303781:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23584768 -> 23588864 00010000:00010000:3.0:1789760737.303801:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0539c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.303817:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0539c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.303828:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0539c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.303836:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760737.303842:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0539c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.303858:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0539c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.303882:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0539c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.303890:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0539c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.304057:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493718912:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52182us (52521us total) trans 4294973077 rc 0/0 00000100:00100000:3.0:1789760737.304068:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5788 00000100:00100000:3.0:1789760737.304080:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747555033350 time 747592891050 state 0x2 ref 0 00000100:00100000:2.0:1789760737.307188:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760737.307279:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493719040 00000100:00100000:3.0:1789760737.307292:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493719040 00000100:00100000:3.0:1789760737.307359:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747555033350 time 747592891050 state 0x0 ref 1 00000100:00100000:3.0:1789760737.307376:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747593151466 time 747592891050 state 0x1 ref 1 00000100:00100000:3.0:1789760737.307410:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747593151466 time 747648445575 state 0x0 ref 1 00000100:00100000:3.0:1789760737.307434:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493719040:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760737.307493:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23588864, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb807 00010000:00010000:3.0:1789760737.307503:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e053a3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.307532:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:3.0:1789760737.307537:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.307545:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e053a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.307566:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e053a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760737.352466:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973078 is committed 00002000:00010000:0.0:1789760737.352697:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23588864 -> 23592960 00010000:00010000:0.0:1789760737.352716:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e053a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.352730:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e053a3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.352740:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e053a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.352748:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760737.352753:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e053a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.352762:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e053a3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.352784:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e053a3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.352791:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e053a3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760737.352911:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493719040:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45482us (45728us total) trans 4294973078 rc 0/0 00000100:00100000:0.0:1789760737.352923:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5789 00000100:00100000:0.0:1789760737.352935:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747593151466 time 747648445575 state 0x2 ref 0 00000100:00100000:2.0:1789760737.356231:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760737.356324:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493719168 00000100:00100000:1.0:1789760737.356339:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493719168 00000100:00100000:1.0:1789760737.356414:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747593151466 time 747648445575 state 0x0 ref 1 00000100:00100000:1.0:1789760737.356435:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747648705991 time 747648445575 state 0x1 ref 1 00000100:00100000:1.0:1789760737.356505:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747648705991 time 747697539363 state 0x0 ref 1 00000100:00100000:1.0:1789760737.356531:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493719168:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760737.356671:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23592960, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb807 00010000:00010000:1.0:1789760737.356681:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e053aa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.356702:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760737.356708:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760737.356715:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e053aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.356732:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e053aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.385424:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760737.385465:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760737.385491:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760737.385540:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493719424 00000100:00100000:2.0:1789760737.385552:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493719424 00000100:00100000:2.0:1789760737.385630:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493719552 00000100:00100000:2.0:1789760737.385634:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493719552 00000100:00100000:1.0:1789760737.385639:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 477 00000100:00100000:1.0:1789760737.385652:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2368:x1876699493719424:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:2.0:1789760737.385661:0:9460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 478 00000100:00100000:2.0:1789760737.385667:0:9460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2368:x1876699493719552:12345-192.168.203.60@tcp:400:kworker.0 00000100:00080000:0.0:1789760737.387375:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760737.387406:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493719296 00000100:00100000:0.0:1789760737.387417:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493719296 00000100:00100000:0.0:1789760737.387481:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 397 00000100:00100000:0.0:1789760737.387493:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d4cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2368:x1876699493719296:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:1.0:1789760737.387582:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2368:x1876699493719424:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 1929us (2170us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760737.387593:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 477 00000100:00100000:0.0:1789760737.387674:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d4cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2368:x1876699493719296:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 183us (2208us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760737.387682:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 397 00000100:00100000:2.0:1789760737.390386:0:9460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2368:x1876699493719552:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 4700us (4875us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760737.390405:0:9460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 478 00000001:00080000:0.0:1789760737.393934:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973079 is committed 00002000:00010000:2.0:1789760737.394130:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23592960 -> 23597056 00010000:00010000:2.0:1789760737.394150:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e053aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.394166:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e053aa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.394177:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e053aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.394191:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760737.394197:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e053aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.394208:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e053aa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.394234:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e053aa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.394243:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e053aa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.394420:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493719168:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37884us (38197us total) trans 4294973079 rc 0/0 00000100:00100000:2.0:1789760737.394433:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5790 00000100:00100000:2.0:1789760737.394447:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747648705991 time 747697539363 state 0x2 ref 0 00000100:00100000:1.0:1789760737.398149:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760737.398252:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493719680 00000100:00100000:2.0:1789760737.398265:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493719680 00000100:00100000:2.0:1789760737.398345:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747648705991 time 747697539363 state 0x0 ref 1 00000100:00100000:2.0:1789760737.398364:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747697799779 time 747697539363 state 0x1 ref 1 00000100:00100000:2.0:1789760737.398433:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747697799779 time 747739467706 state 0x0 ref 1 00000100:00100000:2.0:1789760737.398451:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493719680:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760737.398529:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23597056, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb807 00010000:00010000:2.0:1789760737.398540:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e053b1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.398560:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760737.398574:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.398582:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e053b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.398599:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e053b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760737.454399:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23597056 -> 23601152 00010000:00010000:2.0:1789760737.454428:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e053b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.454445:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e053b1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.454455:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e053b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.454464:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760737.454470:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e053b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.454483:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e053b1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.454519:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e053b1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.454528:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e053b1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.454662:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493719680:12345-192.168.203.60@tcp:4:dd.0 Request processed in 56206us (56701us total) trans 4294973080 rc 0/0 00000100:00100000:2.0:1789760737.454675:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5791 00000100:00100000:2.0:1789760737.454689:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747697799779 time 747739467706 state 0x2 ref 0 00000100:00100000:2.0:1789760737.458633:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760737.458836:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493719808 00000100:00100000:3.0:1789760737.458850:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493719808 00000100:00100000:3.0:1789760737.458922:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747697799779 time 747739467706 state 0x0 ref 1 00000100:00100000:3.0:1789760737.458952:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747739728122 time 747739467706 state 0x1 ref 1 00000100:00100000:2.0:1789760737.459749:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747739728122 time 747800774460 state 0x0 ref 1 00000100:00100000:2.0:1789760737.459779:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493719808:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760737.459854:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23601152, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb807 00010000:00010000:2.0:1789760737.459865:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e053b8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.459885:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760737.459890:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.459899:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e053b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.459918:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e053b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760737.460877:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973080 is committed 00000001:00080000:3.0:1789760737.494423:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973081 is committed 00002000:00010000:2.0:1789760737.494554:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23601152 -> 23605248 00010000:00010000:2.0:1789760737.494583:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e053b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.494597:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e053b8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.494607:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e053b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.494615:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760737.494620:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e053b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.494630:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e053b8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.494655:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e053b8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.494664:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e053b8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.494798:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493719808:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35024us (36172us total) trans 4294973081 rc 0/0 00000100:00100000:2.0:1789760737.494810:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5792 00000100:00100000:2.0:1789760737.494825:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747739728122 time 747800774460 state 0x2 ref 0 00000100:00100000:2.0:1789760737.498822:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760737.498918:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493719936 00000100:00100000:0.0:1789760737.498932:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493719936 00000100:00100000:0.0:1789760737.499249:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747739728122 time 747800774460 state 0x0 ref 1 00000100:00100000:0.0:1789760737.499268:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747801034876 time 747800774460 state 0x1 ref 1 00000100:00100000:0.0:1789760737.499324:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747801034876 time 747840359278 state 0x0 ref 1 00000100:00100000:0.0:1789760737.499339:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493719936:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760737.499400:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23605248, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb807 00010000:00010000:0.0:1789760737.499411:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e053bf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.499431:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:0.0:1789760737.499437:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760737.499444:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e053bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.499463:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e053bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760737.542237:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23605248 -> 23609344 00010000:00010000:0.0:1789760737.542266:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e053bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.542283:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e053bf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.542295:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e053bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.542304:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760737.542311:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e053bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.542321:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e053bf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.542346:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e053bf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.542354:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e053bf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760737.542488:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493719936:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43143us (43673us total) trans 4294973082 rc 0/0 00000100:00100000:0.0:1789760737.542501:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5793 00000100:00100000:0.0:1789760737.542525:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747801034876 time 747840359278 state 0x2 ref 0 00000001:00080000:3.0:1789760737.542675:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973082 is committed 00000100:00100000:0.0:1789760737.545483:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760737.545607:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493720064 00000100:00100000:3.0:1789760737.545621:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493720064 00000100:00100000:3.0:1789760737.545701:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747801034876 time 747840359278 state 0x0 ref 1 00000100:00100000:3.0:1789760737.545722:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747840619694 time 747840359278 state 0x1 ref 1 00000100:00100000:2.0:1789760737.545952:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747840619694 time 747886983008 state 0x0 ref 1 00000100:00100000:2.0:1789760737.545992:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493720064:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760737.546063:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23609344, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb807 00010000:00010000:2.0:1789760737.546075:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e053c6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.546097:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760737.546103:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.546111:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e053c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.546128:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e053c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760737.606525:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973083 is committed 00002000:00010000:2.0:1789760737.606616:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23609344 -> 23613440 00010000:00010000:2.0:1789760737.606631:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e053c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.606650:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e053c6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.606659:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e053c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.606667:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760737.606671:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e053c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.606681:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e053c6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.606701:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e053c6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.606708:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e053c6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.606805:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493720064:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60816us (61331us total) trans 4294973083 rc 0/0 00000100:00100000:2.0:1789760737.606815:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5794 00000100:00100000:2.0:1789760737.606825:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747840619694 time 747886983008 state 0x2 ref 0 00000100:00100000:1.0:1789760737.609630:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760737.609683:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493720192 00000100:00100000:2.0:1789760737.609693:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493720192 00000100:00100000:2.0:1789760737.609745:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747840619694 time 747886983008 state 0x0 ref 1 00000100:00100000:2.0:1789760737.609759:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747887243424 time 747886983008 state 0x1 ref 1 00000100:00100000:2.0:1789760737.609791:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747887243424 time 747950826795 state 0x0 ref 1 00000100:00100000:2.0:1789760737.609805:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493720192:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760737.609852:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23613440, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xb807 00010000:00010000:2.0:1789760737.609860:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e053cd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.609890:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760737.609895:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.609901:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e053cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.609925:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e053cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760737.682218:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973084 is committed 00002000:00010000:1.0:1789760737.682681:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23613440 -> 23617536 00002000:00010000:1.0:1789760737.682687:0:13038:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760736 -> 1789760737 00002000:00010000:1.0:1789760737.682691:0:13038:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760736 -> 1789760737 00010000:00010000:1.0:1789760737.682710:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e053cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.682725:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e053cd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.682735:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e053cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.682744:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760737.682749:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e053cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.682759:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e053cd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.682790:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e053cd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.682799:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e053cd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760737.683077:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493720192:12345-192.168.203.60@tcp:4:dd.0 Request processed in 73267us (73449us total) trans 4294973084 rc 0/0 00000100:00100000:1.0:1789760737.683091:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5795 00000100:00100000:1.0:1789760737.683102:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747887243424 time 747950826795 state 0x2 ref 0 00000100:00100000:3.0:1789760737.691572:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760737.691786:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493720320 00000100:00100000:0.0:1789760737.691799:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493720320 00000100:00100000:0.0:1789760737.691872:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747887243424 time 747950826795 state 0x0 ref 1 00000100:00100000:0.0:1789760737.691890:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747951087211 time 747950826795 state 0x1 ref 1 00000100:00100000:0.0:1789760737.691924:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747951087211 time 748032959764 state 0x0 ref 1 00000100:00100000:0.0:1789760737.691936:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493720320:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760737.692058:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23617536, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:0.0:1789760737.692069:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e053d4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.692089:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:0.0:1789760737.692094:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760737.692101:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e053d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.692117:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e053d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760737.767765:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23617536 -> 23621632 00010000:00010000:2.0:1789760737.767790:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e053d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.767805:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e053d4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.767822:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e053d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.767831:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760737.767837:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e053d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.767846:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e053d4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.767869:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e053d4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.767877:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e053d4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.768047:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493720320:12345-192.168.203.60@tcp:4:dd.0 Request processed in 76105us (76490us total) trans 4294973085 rc 0/0 00000100:00100000:2.0:1789760737.768154:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5796 00000100:00100000:2.0:1789760737.768167:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747951087211 time 748032959764 state 0x2 ref 0 00000001:00080000:1.0:1789760737.768476:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973085 is committed 00000100:00100000:0.0:1789760737.774098:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760737.775806:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493720448 00000100:00100000:3.0:1789760737.775828:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493720448 00000100:00100000:3.0:1789760737.775932:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 747951087211 time 748032959764 state 0x0 ref 1 00000100:00100000:3.0:1789760737.775955:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748033220180 time 748032959764 state 0x1 ref 1 00000100:00100000:3.0:1789760737.776059:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748033220180 time 748117093685 state 0x0 ref 1 00000100:00100000:3.0:1789760737.776077:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493720448:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760737.776148:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23621632, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:3.0:1789760737.776160:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e053db lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.776182:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:3.0:1789760737.776188:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.776196:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e053db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.776215:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e053db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760737.834352:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973086 is committed 00002000:00010000:3.0:1789760737.834519:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23621632 -> 23625728 00010000:00010000:3.0:1789760737.834539:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e053db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.834555:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e053db lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.834575:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e053db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.834585:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760737.834591:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e053db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.834601:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e053db lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.834625:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e053db lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.834634:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e053db lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.834763:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493720448:12345-192.168.203.60@tcp:4:dd.0 Request processed in 58682us (60672us total) trans 4294973086 rc 0/0 00000100:00100000:3.0:1789760737.834776:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5797 00000100:00100000:3.0:1789760737.834907:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748033220180 time 748117093685 state 0x2 ref 0 00000100:00100000:2.0:1789760737.840580:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760737.840729:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493720576 00000100:00100000:0.0:1789760737.840742:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493720576 00000100:00100000:0.0:1789760737.840934:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748033220180 time 748117093685 state 0x0 ref 1 00000100:00100000:0.0:1789760737.840954:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748117354101 time 748117093685 state 0x1 ref 1 00000100:00100000:2.0:1789760737.841882:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748117354101 time 748182910846 state 0x0 ref 1 00000100:00100000:2.0:1789760737.841912:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493720576:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760737.842103:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23625728, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:2.0:1789760737.842121:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e053e2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.842230:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760737.842236:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.842244:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e053e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.842262:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e053e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.856726:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760737.928377:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973087 is committed 00002000:00010000:3.0:1789760737.928921:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23625728 -> 23629824 00010000:00010000:3.0:1789760737.928943:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e053e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.928988:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e053e2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.929005:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e053e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.929013:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760737.929020:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e053e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.929029:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e053e2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.929053:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e053e2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.929061:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e053e2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.929176:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493720576:12345-192.168.203.60@tcp:4:dd.0 Request processed in 87269us (88606us total) trans 4294973087 rc 0/0 00000100:00100000:3.0:1789760737.929187:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5798 00000100:00100000:3.0:1789760737.929200:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748117354101 time 748182910846 state 0x2 ref 0 00000100:00100000:2.0:1789760737.938130:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760737.938266:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493720704 00000100:00100000:1.0:1789760737.938281:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493720704 00000100:00100000:1.0:1789760737.938428:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748117354101 time 748182910846 state 0x0 ref 1 00000100:00100000:1.0:1789760737.938449:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748183171262 time 748182910846 state 0x1 ref 1 00000100:00100000:1.0:1789760737.938485:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748183171262 time 748279520626 state 0x0 ref 1 00000100:00100000:1.0:1789760737.938502:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493720704:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760737.938652:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23629824, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:1.0:1789760737.938663:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e053e9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.938720:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760737.938726:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760737.938734:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e053e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760737.938752:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e053e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760737.991813:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973088 is committed 00002000:00010000:2.0:1789760737.991955:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23629824 -> 23633920 00010000:00010000:2.0:1789760737.992101:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e053e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.992116:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e053e9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.992126:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e053e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.992140:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760737.992145:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e053e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.992155:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e053e9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.992178:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e053e9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.992185:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e053e9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.992355:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493720704:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53849us (54233us total) trans 4294973088 rc 0/0 00000100:00100000:2.0:1789760737.992369:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5799 00000100:00100000:2.0:1789760737.992382:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748183171262 time 748279520626 state 0x2 ref 0 00000100:00100000:0.0:1789760737.996216:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760737.996310:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493720832 00000100:00100000:2.0:1789760737.996323:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493720832 00000100:00100000:2.0:1789760737.996398:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748183171262 time 748279520626 state 0x0 ref 1 00000100:00100000:2.0:1789760737.996416:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748279781042 time 748279520626 state 0x1 ref 1 00000100:00100000:2.0:1789760737.996567:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748279781042 time 748337590416 state 0x0 ref 1 00000100:00100000:2.0:1789760737.996589:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493720832:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760737.996861:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23633920, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:2.0:1789760737.996872:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e053f0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.996892:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760737.996897:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.996904:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e053f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.996920:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e053f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760738.048380:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973089 is committed 00002000:00010000:2.0:1789760738.048681:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23633920 -> 23638016 00010000:00010000:2.0:1789760738.048700:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e053f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.048715:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e053f0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.048724:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e053f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.048732:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760738.048737:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e053f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.048747:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e053f0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.048769:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e053f0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.048777:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e053f0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.048903:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493720832:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52310us (52693us total) trans 4294973089 rc 0/0 00000100:00100000:2.0:1789760738.048916:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5800 00000100:00100000:2.0:1789760738.048928:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748279781042 time 748337590416 state 0x2 ref 0 00000100:00100000:1.0:1789760738.053585:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760738.053881:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493720960 00000100:00100000:3.0:1789760738.053894:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493720960 00000100:00100000:3.0:1789760738.055149:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748279781042 time 748337590416 state 0x0 ref 1 00000100:00100000:3.0:1789760738.055174:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748337850832 time 748337590416 state 0x1 ref 1 00000100:00100000:1.0:1789760738.055372:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748337850832 time 748396404918 state 0x0 ref 1 00000100:00100000:1.0:1789760738.055399:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493720960:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760738.055482:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23638016, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:1.0:1789760738.055492:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e053f7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.055520:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760738.055526:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.055533:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e053f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.055549:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e053f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760738.105610:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973090 is committed 00002000:00010000:1.0:1789760738.105778:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23638016 -> 23642112 00010000:00010000:1.0:1789760738.105802:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e053f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.105818:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e053f7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.105828:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e053f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.105836:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760738.105842:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e053f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.105853:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e053f7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.105880:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e053f7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.105899:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e053f7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760738.106761:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493720960:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51365us (53194us total) trans 4294973090 rc 0/0 00000100:00100000:1.0:1789760738.106777:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5801 00000100:00100000:1.0:1789760738.106790:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748337850832 time 748396404918 state 0x2 ref 0 00000100:00100000:3.0:1789760738.111038:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760738.111130:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493721088 00000100:00100000:2.0:1789760738.111146:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493721088 00000100:00100000:2.0:1789760738.111201:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748337850832 time 748396404918 state 0x0 ref 1 00000100:00100000:2.0:1789760738.111216:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748396665334 time 748396404918 state 0x1 ref 1 00000100:00100000:0.0:1789760738.111261:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748396665334 time 748452293253 state 0x0 ref 1 00000100:00100000:0.0:1789760738.111276:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493721088:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760738.111329:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23642112, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:0.0:1789760738.111340:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e053fe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.111355:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:0.0:1789760738.111360:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.111366:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e053fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.111380:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e053fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760738.156231:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973091 is committed 00002000:00010000:0.0:1789760738.156426:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23642112 -> 23646208 00010000:00010000:0.0:1789760738.156447:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e053fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.156462:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e053fe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.156473:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e053fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.156480:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760738.156486:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e053fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.156498:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e053fe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.156540:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e053fe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.156549:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e053fe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.156747:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493721088:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45467us (45713us total) trans 4294973091 rc 0/0 00000100:00100000:0.0:1789760738.156760:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5802 00000100:00100000:0.0:1789760738.156773:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748396665334 time 748452293253 state 0x2 ref 0 00000100:00100000:2.0:1789760738.159619:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760738.159747:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493721216 00000100:00100000:1.0:1789760738.159761:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493721216 00000100:00100000:1.0:1789760738.159833:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748396665334 time 748452293253 state 0x0 ref 1 00000100:00100000:1.0:1789760738.159853:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748452553669 time 748452293253 state 0x1 ref 1 00000100:00100000:1.0:1789760738.159995:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748452553669 time 748500997098 state 0x0 ref 1 00000100:00100000:1.0:1789760738.160013:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493721216:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760738.160076:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23646208, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:1.0:1789760738.160087:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05405 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.160107:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760738.160114:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.160122:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05405 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.160139:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05405 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.170526:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc5836300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3303:1876699519251840:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760738.170637:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760738.170844:0:6691:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519251840 00000100:00100000:3.0:1789760738.170855:0:6691:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519251840 00000100:00100000:3.0:1789760738.171030:0:6691:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 198 00000100:00100000:3.0:1789760738.171041:0:6691:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876699519251840:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789760738.171070:0:6691:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528023552 pending 1703936 free 3834642432 avail 3806330880 00000100:00100000:3.0:1789760738.171129:0:6691:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876699519251840:12345-0@lo:13:osp-pre-0-0.0 Request processed in 90us (492us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760738.171137:0:6691:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 198 00000100:00100000:3.0:1789760738.171194:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc5836300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3303:1876699519251840:0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1789760738.171781:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760738.192258:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973092 is committed 00002000:00010000:2.0:1789760738.192350:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23646208 -> 23650304 00010000:00010000:2.0:1789760738.192371:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05405 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.192387:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05405 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.192397:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05405 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.192405:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760738.192410:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05405 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.192420:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05405 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.192444:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05405 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.192452:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05405 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.192700:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493721216:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32684us (33089us total) trans 4294973092 rc 0/0 00000100:00100000:2.0:1789760738.192712:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5803 00000100:00100000:2.0:1789760738.192724:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748452553669 time 748500997098 state 0x2 ref 0 00000100:00100000:1.0:1789760738.204699:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760738.204785:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493721344 00000100:00100000:3.0:1789760738.204865:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493721344 00000100:00100000:3.0:1789760738.204943:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748452553669 time 748500997098 state 0x0 ref 1 00000100:00100000:3.0:1789760738.205081:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748501257514 time 748500997098 state 0x1 ref 1 00000100:00100000:3.0:1789760738.205110:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748501257514 time 748546145879 state 0x0 ref 1 00000100:00100000:3.0:1789760738.205126:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493721344:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760738.205247:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23650304, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:3.0:1789760738.205258:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0540c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.205277:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:3.0:1789760738.205283:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.205291:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0540c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.205308:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0540c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760738.256158:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973093 is committed 00002000:00010000:0.0:1789760738.256317:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23650304 -> 23654400 00010000:00010000:0.0:1789760738.256332:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0540c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.256345:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0540c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.256354:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0540c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.256361:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760738.256366:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0540c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.256381:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0540c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.256400:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0540c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.256407:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0540c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.256682:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493721344:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51555us (51993us total) trans 4294973093 rc 0/0 00000100:00100000:0.0:1789760738.256693:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5804 00000100:00100000:0.0:1789760738.256703:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748501257514 time 748546145879 state 0x2 ref 0 00000100:00100000:3.0:1789760738.267954:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760738.268131:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493721472 00000100:00100000:2.0:1789760738.268141:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493721472 00000100:00100000:2.0:1789760738.268198:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748501257514 time 748546145879 state 0x0 ref 1 00000100:00100000:2.0:1789760738.268214:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748546406295 time 748546145879 state 0x1 ref 1 00000100:00100000:2.0:1789760738.268247:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748546406295 time 748609283029 state 0x0 ref 1 00000100:00100000:2.0:1789760738.268260:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493721472:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760738.268310:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23654400, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:2.0:1789760738.268319:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05413 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.268441:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760738.268446:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.268453:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05413 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.268707:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05413 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760738.305525:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973094 is committed 00002000:00010000:2.0:1789760738.305596:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23654400 -> 23658496 00010000:00010000:2.0:1789760738.305611:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05413 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.305625:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05413 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.305631:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05413 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.305635:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760738.305639:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05413 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.305646:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05413 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.305662:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05413 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.305666:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05413 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.305778:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493721472:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37515us (37827us total) trans 4294973094 rc 0/0 00000100:00100000:2.0:1789760738.305789:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5805 00000100:00100000:2.0:1789760738.305799:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748546406295 time 748609283029 state 0x2 ref 0 00000100:00100000:3.0:1789760738.311251:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760738.311369:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493721600 00000100:00100000:0.0:1789760738.311379:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493721600 00000100:00100000:0.0:1789760738.311430:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748546406295 time 748609283029 state 0x0 ref 1 00000100:00100000:0.0:1789760738.311446:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748609543445 time 748609283029 state 0x1 ref 1 00000100:00100000:0.0:1789760738.311477:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748609543445 time 748652512877 state 0x0 ref 1 00000100:00100000:0.0:1789760738.311490:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493721600:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760738.311546:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23658496, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:0.0:1789760738.311564:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0541a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.311581:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:0.0:1789760738.311586:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.311593:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0541a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.311607:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0541a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760738.364592:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23658496 -> 23662592 00010000:00010000:2.0:1789760738.364612:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0541a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.364624:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0541a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.364632:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0541a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.364640:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760738.364644:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0541a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.364653:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0541a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.364670:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0541a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.364678:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0541a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.364775:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493721600:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53282us (53528us total) trans 4294973095 rc 0/0 00000100:00100000:2.0:1789760738.364786:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5806 00000100:00100000:2.0:1789760738.364795:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748609543445 time 748652512877 state 0x2 ref 0 00000001:00080000:3.0:1789760738.365474:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973095 is committed 00000100:00100000:0.0:1789760738.368854:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760738.368946:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493721728 00000100:00100000:1.0:1789760738.368955:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493721728 00000100:00100000:2.0:1789760738.369301:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748609543445 time 748652512877 state 0x0 ref 1 00000100:00100000:2.0:1789760738.369317:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748652773293 time 748652512877 state 0x1 ref 1 00000100:00100000:2.0:1789760738.369348:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748652773293 time 748710383718 state 0x0 ref 1 00000100:00100000:2.0:1789760738.369361:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493721728:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760738.369430:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23662592, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:2.0:1789760738.369440:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05421 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.369455:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760738.369460:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.369466:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05421 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.369481:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05421 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760738.396339:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23662592 -> 23666688 00010000:00010000:2.0:1789760738.396359:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05421 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.396374:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05421 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.396390:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05421 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.396398:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760738.396403:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05421 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.396414:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05421 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.396433:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05421 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.396441:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05421 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.396535:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493721728:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27172us (27686us total) trans 4294973096 rc 0/0 00000100:00100000:2.0:1789760738.396546:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5807 00000100:00100000:2.0:1789760738.396659:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748652773293 time 748710383718 state 0x2 ref 0 00000001:00080000:2.0:1789760738.396698:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973096 is committed 00000100:00100000:0.0:1789760738.400250:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760738.400424:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493721856 00000100:00100000:3.0:1789760738.400433:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493721856 00000100:00100000:3.0:1789760738.400486:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748652773293 time 748710383718 state 0x0 ref 1 00000100:00100000:3.0:1789760738.400785:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748710644134 time 748710383718 state 0x1 ref 1 00000100:00100000:3.0:1789760738.400847:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748710644134 time 748741881842 state 0x0 ref 1 00000100:00100000:3.0:1789760738.400862:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493721856:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760738.400908:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23666688, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:3.0:1789760738.401229:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05428 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.401245:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:3.0:1789760738.401249:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.401256:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05428 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.401277:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05428 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760738.474379:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973097 is committed 00002000:00010000:3.0:1789760738.474517:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23666688 -> 23670784 00010000:00010000:3.0:1789760738.474532:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05428 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.474544:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05428 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.474616:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05428 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.474625:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760738.474630:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05428 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.474639:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05428 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.474659:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05428 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.474666:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05428 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.474803:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493721856:12345-192.168.203.60@tcp:4:dd.0 Request processed in 73939us (74557us total) trans 4294973097 rc 0/0 00000100:00100000:3.0:1789760738.474813:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5808 00000100:00100000:3.0:1789760738.474822:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748710644134 time 748741881842 state 0x2 ref 0 00000100:00100000:2.0:1789760738.477901:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760738.478099:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493721984 00000100:00100000:1.0:1789760738.478109:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493721984 00000100:00100000:1.0:1789760738.478166:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748710644134 time 748741881842 state 0x0 ref 1 00000100:00100000:1.0:1789760738.478182:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748742142258 time 748741881842 state 0x1 ref 1 00000100:00100000:1.0:1789760738.478243:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748742142258 time 748819277993 state 0x0 ref 1 00000100:00100000:1.0:1789760738.478256:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493721984:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760738.478309:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23670784, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:1.0:1789760738.478318:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0542f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.478334:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760738.478338:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.478344:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0542f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.478362:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0542f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760738.502731:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973098 is committed 00002000:00010000:0.0:1789760738.502894:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23670784 -> 23674880 00010000:00010000:0.0:1789760738.502909:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0542f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.503159:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0542f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.503167:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0542f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.503176:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760738.503180:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0542f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.503190:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0542f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.503334:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0542f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.503344:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0542f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.503439:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493721984:12345-192.168.203.60@tcp:4:dd.0 Request processed in 25180us (25542us total) trans 4294973098 rc 0/0 00000100:00100000:0.0:1789760738.503450:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5809 00000100:00100000:0.0:1789760738.503460:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748742142258 time 748819277993 state 0x2 ref 0 00000100:00100000:2.0:1789760738.507054:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760738.507097:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493722112 00000100:00100000:0.0:1789760738.507108:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493722112 00000100:00100000:0.0:1789760738.507169:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748742142258 time 748819277993 state 0x0 ref 1 00000100:00100000:0.0:1789760738.507184:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748819538409 time 748819277993 state 0x1 ref 1 00000100:00100000:0.0:1789760738.507284:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748819538409 time 748848252333 state 0x0 ref 1 00000100:00100000:0.0:1789760738.507299:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493722112:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760738.507347:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23674880, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:0.0:1789760738.507356:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05436 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.507371:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:0.0:1789760738.507376:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.507382:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05436 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.507396:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05436 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760738.582326:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973099 is committed 00002000:00010000:1.0:1789760738.582460:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23674880 -> 23678976 00010000:00010000:1.0:1789760738.582477:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05436 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.582489:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05436 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.582497:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05436 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.582513:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760738.582518:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05436 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.582526:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05436 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.582545:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05436 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.582560:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05436 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760738.582657:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493722112:12345-192.168.203.60@tcp:4:dd.0 Request processed in 75356us (75605us total) trans 4294973099 rc 0/0 00000100:00100000:1.0:1789760738.582667:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5810 00000100:00100000:1.0:1789760738.582677:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748819538409 time 748848252333 state 0x2 ref 0 00000100:00100000:3.0:1789760738.587634:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760738.587782:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493722240 00000100:00100000:2.0:1789760738.587793:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493722240 00000100:00100000:2.0:1789760738.587854:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748819538409 time 748848252333 state 0x0 ref 1 00000100:00100000:2.0:1789760738.587870:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748848512749 time 748848252333 state 0x1 ref 1 00000100:00100000:2.0:1789760738.587901:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748848512749 time 748928937212 state 0x0 ref 1 00000100:00100000:2.0:1789760738.587915:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493722240:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760738.587984:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23678976, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xb807 00010000:00010000:2.0:1789760738.587994:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0543d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.588010:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760738.588015:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.588022:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0543d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.588035:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0543d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760738.630894:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23678976 -> 23683072 00002000:00010000:1.0:1789760738.630905:0:13038:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760737 -> 1789760738 00002000:00010000:1.0:1789760738.630908:0:13038:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760737 -> 1789760738 00010000:00010000:1.0:1789760738.630922:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0543d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.630935:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0543d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.630943:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0543d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.630952:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760738.630957:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0543d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.630993:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0543d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.631012:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0543d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.631020:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0543d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760738.631105:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493722240:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43189us (43476us total) trans 4294973100 rc 0/0 00000100:00100000:1.0:1789760738.631115:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5811 00000100:00100000:1.0:1789760738.631126:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748848512749 time 748928937212 state 0x2 ref 0 00000001:00080000:2.0:1789760738.631218:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973100 is committed 00000100:00100000:0.0:1789760738.635166:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760738.635409:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493722368 00000100:00100000:3.0:1789760738.635427:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493722368 00000100:00100000:3.0:1789760738.635515:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748848512749 time 748928937212 state 0x0 ref 1 00000100:00100000:3.0:1789760738.635534:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748929197628 time 748928937212 state 0x1 ref 1 00000100:00100000:3.0:1789760738.635641:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748929197628 time 748976674361 state 0x0 ref 1 00000100:00100000:3.0:1789760738.635661:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493722368:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760738.635738:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23683072, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:3.0:1789760738.635750:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05444 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.635876:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=748s v=(0 0) 00010000:00010000:3.0:1789760738.635883:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.635891:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05444 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.635911:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05444 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760738.670170:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23683072 -> 23687168 00010000:00010000:0.0:1789760738.670200:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05444 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.670216:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05444 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.670226:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05444 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.670235:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760738.670240:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05444 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.670252:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05444 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.670284:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05444 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.670308:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05444 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.670439:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493722368:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34776us (35281us total) trans 4294973101 rc 0/0 00000100:00100000:0.0:1789760738.670453:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5812 00000100:00100000:0.0:1789760738.670466:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748929197628 time 748976674361 state 0x2 ref 0 00000001:00080000:1.0:1789760738.673182:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973101 is committed 00000100:00100000:3.0:1789760738.674638:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760738.674801:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493722496 00000100:00100000:0.0:1789760738.674816:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493722496 00000100:00100000:0.0:1789760738.674892:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748929197628 time 748976674361 state 0x0 ref 1 00000100:00100000:0.0:1789760738.674911:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748976934777 time 748976674361 state 0x1 ref 1 00000100:00100000:0.0:1789760738.674937:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748976934777 time 749015972677 state 0x0 ref 1 00000100:00100000:0.0:1789760738.674951:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493722496:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760738.675064:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23687168, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:0.0:1789760738.675076:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0544b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.675097:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760738.675103:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.675110:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0544b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.675134:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0544b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.691152:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc33ec000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3304:1876699519251968:0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.691252:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760738.691313:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc33eca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3304:1876699519252480:0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.691423:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760738.691461:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760738.691467:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc33ed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3304:1876699519252352:0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.691495:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760738.691544:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc33ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3305:1876699519252096:0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.691635:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760738.691656:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc33ed500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519252608:0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.691683:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760738.691704:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789760738.691711:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc33eea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519252224:0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.691733:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760738.691875:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519251968 00000100:00100000:3.0:1789760738.691886:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519251968 00000100:00100000:3.0:1789760738.691944:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 398 00000100:00100000:3.0:1789760738.691955:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d4fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3304:x1876699519251968:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.692042:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d4fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3304:x1876699519251968:12345-0@lo:400:kworker.0 Request processed in 89us (790us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760738.692051:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 398 00000100:00100000:3.0:1789760738.692092:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519252480 00000100:00100000:3.0:1789760738.692096:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519252480 00000100:00100000:3.0:1789760738.692137:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519252352 00000100:00100000:3.0:1789760738.692140:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519252352 00000100:00100000:3.0:1789760738.692159:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519252096 00000100:00100000:3.0:1789760738.692162:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519252096 00000100:00100000:3.0:1789760738.692177:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 479 00000100:00100000:3.0:1789760738.692184:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ee300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3304:x1876699519252480:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.692220:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ee300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3304:x1876699519252480:12345-0@lo:400:kworker.0 Request processed in 36us (796us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760738.692227:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 479 00000100:00100000:3.0:1789760738.692240:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 480 00000100:00100000:3.0:1789760738.692247:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3304:x1876699519252352:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.692275:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3304:x1876699519252352:12345-0@lo:400:kworker.0 Request processed in 29us (780us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760738.692281:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 480 00000100:00100000:3.0:1789760738.692292:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 481 00000100:00100000:3.0:1789760738.692297:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3305:x1876699519252096:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.692425:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3305:x1876699519252096:12345-0@lo:400:kworker.0 Request processed in 127us (789us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760738.692432:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 481 00000100:00100000:3.0:1789760738.692463:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519252608 00000100:00100000:3.0:1789760738.692467:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519252608 00000100:00100000:3.0:1789760738.692510:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519252224 00000100:00100000:3.0:1789760738.692513:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519252224 00000100:00100000:3.0:1789760738.692536:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 172 00000100:00100000:3.0:1789760738.692543:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519252608:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.692650:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519252608:12345-0@lo:400:kworker.0 Request processed in 107us (966us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760738.692660:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 172 00000100:00100000:3.0:1789760738.692672:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 173 00000100:00100000:3.0:1789760738.692678:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519252224:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.692706:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519252224:12345-0@lo:400:kworker.0 Request processed in 28us (971us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760738.692712:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 173 00000100:00080000:3.0:1789760738.692739:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789760738.692745:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760738.692818:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc33eea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519252224:0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.692833:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc33ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3305:1876699519252096:0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.692844:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc33ed500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519252608:0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.692887:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc33ed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3304:1876699519252352:0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.692898:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc33ec000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3304:1876699519251968:0@lo:400:kworker.0 00000100:00100000:3.0:1789760738.692908:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc33eca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3304:1876699519252480:0@lo:400:kworker.0 00002000:00010000:0.0:1789760738.746525:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23687168 -> 23691264 00010000:00010000:0.0:1789760738.746560:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0544b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.746575:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0544b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.746586:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0544b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.746594:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760738.746600:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0544b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.746610:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0544b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.746635:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0544b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.746644:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0544b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.746781:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493722496:12345-192.168.203.60@tcp:4:dd.0 Request processed in 71824us (72146us total) trans 4294973102 rc 0/0 00000100:00100000:0.0:1789760738.746794:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5813 00000100:00100000:0.0:1789760738.746806:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748976934777 time 749015972677 state 0x2 ref 0 00000001:00080000:3.0:1789760738.748368:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973102 is committed 00000100:00100000:2.0:1789760738.755321:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760738.755656:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493722624 00000100:00100000:1.0:1789760738.755668:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493722624 00000100:00100000:1.0:1789760738.755744:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 748976934777 time 749015972677 state 0x0 ref 1 00000100:00100000:1.0:1789760738.755764:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749016233093 time 749015972677 state 0x1 ref 1 00000100:00100000:1.0:1789760738.755800:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749016233093 time 749096835179 state 0x0 ref 1 00000100:00100000:1.0:1789760738.755814:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493722624:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760738.756342:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23691264, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:1.0:1789760738.756356:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05452 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.756377:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760738.756382:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.756390:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05452 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.756410:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05452 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760738.821936:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23691264 -> 23695360 00010000:00010000:1.0:1789760738.821990:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05452 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.822163:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05452 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.822176:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05452 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.822185:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760738.822190:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05452 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.822201:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05452 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.822224:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05452 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.822232:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05452 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760738.822433:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493722624:12345-192.168.203.60@tcp:4:dd.0 Request processed in 66613us (67119us total) trans 4294973103 rc 0/0 00000100:00100000:1.0:1789760738.822446:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5814 00000100:00100000:1.0:1789760738.822458:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749016233093 time 749096835179 state 0x2 ref 0 00000001:00080000:3.0:1789760738.823109:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973103 is committed 00000100:00100000:2.0:1789760738.825924:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760738.826143:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493722752 00000100:00100000:1.0:1789760738.826157:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493722752 00000100:00100000:1.0:1789760738.826237:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749016233093 time 749096835179 state 0x0 ref 1 00000100:00100000:1.0:1789760738.826257:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749097095595 time 749096835179 state 0x1 ref 1 00000100:00100000:1.0:1789760738.826325:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749097095595 time 749167359818 state 0x0 ref 1 00000100:00100000:1.0:1789760738.826339:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493722752:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760738.826403:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23695360, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:1.0:1789760738.826414:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05459 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.826435:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760738.826440:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760738.826448:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05459 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.826592:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05459 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760738.868383:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23695360 -> 23699456 00010000:00010000:1.0:1789760738.868420:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05459 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.868439:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05459 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.868452:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05459 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.868460:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760738.868483:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05459 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.868496:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05459 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.868527:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05459 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.868535:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05459 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760738.868673:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493722752:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42324us (42750us total) trans 4294973104 rc 0/0 00000100:00100000:1.0:1789760738.868691:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5815 00000100:00100000:1.0:1789760738.868703:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749097095595 time 749167359818 state 0x2 ref 0 00000001:00080000:3.0:1789760738.869897:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973104 is committed 00000100:00100000:3.0:1789760738.872941:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760738.873133:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493722880 00000100:00100000:0.0:1789760738.873146:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493722880 00000100:00100000:0.0:1789760738.873221:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749097095595 time 749167359818 state 0x0 ref 1 00000100:00100000:0.0:1789760738.873242:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749167620234 time 749167359818 state 0x1 ref 1 00000100:00100000:0.0:1789760738.873284:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749167620234 time 749214319142 state 0x0 ref 1 00000100:00100000:0.0:1789760738.873298:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493722880:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760738.873368:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23699456, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:0.0:1789760738.873379:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e05460 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.873400:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760738.873405:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.873413:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e05460 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.873429:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e05460 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.882573:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760738.920842:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973105 is committed 00002000:00010000:2.0:1789760738.921022:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23699456 -> 23703552 00010000:00010000:2.0:1789760738.921044:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e05460 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.921060:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e05460 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.921069:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e05460 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.921076:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760738.921081:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e05460 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.921091:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e05460 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.921114:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e05460 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.921122:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e05460 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.921237:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493722880:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47934us (48306us total) trans 4294973105 rc 0/0 00000100:00100000:2.0:1789760738.921249:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5816 00000100:00100000:2.0:1789760738.921261:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749167620234 time 749214319142 state 0x2 ref 0 00000100:00100000:0.0:1789760738.923942:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760738.924168:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493723008 00000100:00100000:2.0:1789760738.924181:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493723008 00000100:00100000:2.0:1789760738.924255:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749167620234 time 749214319142 state 0x0 ref 1 00000100:00100000:2.0:1789760738.924274:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749214579558 time 749214319142 state 0x1 ref 1 00000100:00100000:2.0:1789760738.924311:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749214579558 time 749265346439 state 0x0 ref 1 00000100:00100000:2.0:1789760738.924327:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493723008:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760738.924391:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23703552, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:2.0:1789760738.924402:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05467 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.924423:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760738.924429:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.924437:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05467 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.924454:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05467 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760738.958598:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23703552 -> 23707648 00010000:00010000:2.0:1789760738.958626:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05467 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.958642:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05467 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.958654:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05467 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.958661:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00000001:00080000:1.0:1789760738.958667:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973106 is committed 00010000:00010000:2.0:1789760738.958667:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05467 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.958677:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05467 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.958701:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05467 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.958709:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05467 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.958832:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493723008:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34501us (34897us total) trans 4294973106 rc 0/0 00000100:00100000:2.0:1789760738.958845:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5817 00000100:00100000:2.0:1789760738.958856:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749214579558 time 749265346439 state 0x2 ref 0 00000100:00100000:0.0:1789760738.962461:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760738.962635:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493723136 00000100:00100000:3.0:1789760738.962649:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493723136 00000100:00100000:3.0:1789760738.962723:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749214579558 time 749265346439 state 0x0 ref 1 00000100:00100000:3.0:1789760738.962743:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749265606855 time 749265346439 state 0x1 ref 1 00000100:00100000:3.0:1789760738.962780:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749265606855 time 749303814860 state 0x0 ref 1 00000100:00100000:3.0:1789760738.962794:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493723136:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760738.962862:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23707648, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:3.0:1789760738.962873:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0546e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.962903:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760738.962909:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.962916:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0546e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.962933:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0546e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760739.007324:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23707648 -> 23711744 00010000:00010000:2.0:1789760739.007351:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0546e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.007371:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0546e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.007380:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0546e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.007399:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760739.007405:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0546e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.007415:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0546e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.007453:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0546e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.007461:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0546e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.007604:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493723136:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44805us (45151us total) trans 4294973107 rc 0/0 00000100:00100000:2.0:1789760739.007617:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5818 00000100:00100000:2.0:1789760739.007630:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749265606855 time 749303814860 state 0x2 ref 0 00000001:00080000:1.0:1789760739.007949:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973107 is committed 00000100:00100000:3.0:1789760739.010649:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760739.011238:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493723264 00000100:00100000:0.0:1789760739.011253:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493723264 00000100:00100000:0.0:1789760739.011403:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749265606855 time 749303814860 state 0x0 ref 1 00000100:00100000:0.0:1789760739.011433:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749304075276 time 749303814860 state 0x1 ref 1 00000100:00100000:0.0:1789760739.011468:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749304075276 time 749352503383 state 0x0 ref 1 00000100:00100000:0.0:1789760739.011488:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493723264:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760739.011561:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23711744, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:0.0:1789760739.011571:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05475 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.011592:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760739.011597:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.011605:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05475 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.011621:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05475 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760739.012644:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789760739.059118:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23711744 -> 23715840 00010000:00010000:1.0:1789760739.059145:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05475 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.059157:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05475 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.059164:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05475 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.059169:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760739.059173:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05475 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.059181:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05475 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.059201:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05475 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.059205:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05475 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.059320:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493723264:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47832us (48679us total) trans 4294973108 rc 0/0 00000100:00100000:1.0:1789760739.059333:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5819 00000100:00100000:1.0:1789760739.059345:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749304075276 time 749352503383 state 0x2 ref 0 00000001:00080000:2.0:1789760739.059803:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973108 is committed 00000100:00100000:3.0:1789760739.062807:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760739.063104:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493723392 00000100:00100000:2.0:1789760739.063117:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493723392 00000100:00100000:2.0:1789760739.063197:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749304075276 time 749352503383 state 0x0 ref 1 00000100:00100000:2.0:1789760739.063226:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749352763799 time 749352503383 state 0x1 ref 1 00000100:00100000:2.0:1789760739.063276:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749352763799 time 749404311457 state 0x0 ref 1 00000100:00100000:2.0:1789760739.063291:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493723392:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760739.063359:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23715840, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:2.0:1789760739.063379:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0547c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.063406:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760739.063411:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.063425:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0547c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.063442:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0547c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760739.114893:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23715840 -> 23719936 00010000:00010000:3.0:1789760739.114921:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0547c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.114936:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0547c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.114956:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0547c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.114996:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760739.115002:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0547c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.115013:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0547c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.115039:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0547c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.115048:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0547c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.115176:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493723392:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51881us (52378us total) trans 4294973109 rc 0/0 00000100:00100000:3.0:1789760739.115190:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5820 00000100:00100000:3.0:1789760739.115202:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749352763799 time 749404311457 state 0x2 ref 0 00000001:00080000:3.0:1789760739.116281:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973109 is committed 00000100:00100000:3.0:1789760739.118800:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760739.118998:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493723520 00000100:00100000:2.0:1789760739.119012:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493723520 00000100:00100000:2.0:1789760739.119088:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749352763799 time 749404311457 state 0x0 ref 1 00000100:00100000:2.0:1789760739.119108:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749404571873 time 749404311457 state 0x1 ref 1 00000100:00100000:2.0:1789760739.119144:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749404571873 time 749460179592 state 0x0 ref 1 00000100:00100000:2.0:1789760739.119160:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493723520:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760739.119226:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23719936, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:2.0:1789760739.119238:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05483 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.119256:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760739.119261:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.119269:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05483 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.119285:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05483 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760739.164526:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973110 is committed 00002000:00010000:1.0:1789760739.165046:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23719936 -> 23724032 00010000:00010000:1.0:1789760739.165069:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05483 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.165084:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05483 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.165094:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05483 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.165103:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760739.165109:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05483 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.165120:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05483 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.165144:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05483 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.165158:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05483 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.165292:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493723520:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46127us (46498us total) trans 4294973110 rc 0/0 00000100:00100000:1.0:1789760739.165305:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5821 00000100:00100000:1.0:1789760739.165318:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749404571873 time 749460179592 state 0x2 ref 0 00000100:00100000:0.0:1789760739.169234:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760739.169661:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493723648 00000100:00100000:3.0:1789760739.169676:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493723648 00000100:00100000:3.0:1789760739.169757:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749404571873 time 749460179592 state 0x0 ref 1 00000100:00100000:3.0:1789760739.169776:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749460440008 time 749460179592 state 0x1 ref 1 00000100:00100000:3.0:1789760739.170575:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749460440008 time 749511602891 state 0x0 ref 1 00000100:00100000:3.0:1789760739.171035:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493723648:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760739.171116:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23724032, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:3.0:1789760739.171128:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0548a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.171149:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760739.171153:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.171160:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0548a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.171177:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0548a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.195227:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760739.195252:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc2885c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519252736:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760739.195319:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760739.195382:0:6691:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519252736 00000100:00100000:1.0:1789760739.195388:0:6691:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519252736 00000100:00100000:1.0:1789760739.195433:0:6691:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 199 00000100:00100000:1.0:1789760739.195442:0:6691:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519252736:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760739.195477:0:6691:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760739.195618:0:6691:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519252736:12345-0@lo:13:osp-pre-1-0.0 Request processed in 177us (298us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760739.195626:0:6691:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 199 00000100:00100000:1.0:1789760739.195671:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc2885c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519252736:0@lo:13:osp-pre-1-0.0 00000001:00080000:1.0:1789760739.206437:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973111 is committed 00002000:00010000:0.0:1789760739.206557:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23724032 -> 23728128 00010000:00010000:0.0:1789760739.206572:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0548a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.206585:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0548a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.206594:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0548a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.206601:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760739.206606:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0548a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.206616:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0548a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.206636:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0548a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.206643:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0548a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.206732:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493723648:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35709us (37510us total) trans 4294973111 rc 0/0 00000100:00100000:0.0:1789760739.206741:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5822 00000100:00100000:0.0:1789760739.206751:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749460440008 time 749511602891 state 0x2 ref 0 00000100:00100000:3.0:1789760739.213149:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760739.213225:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493723776 00000100:00100000:1.0:1789760739.213236:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493723776 00000100:00100000:1.0:1789760739.213292:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749460440008 time 749511602891 state 0x0 ref 1 00000100:00100000:1.0:1789760739.213311:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749511863307 time 749511602891 state 0x1 ref 1 00000100:00100000:1.0:1789760739.213351:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749511863307 time 749554386054 state 0x0 ref 1 00000100:00100000:1.0:1789760739.213364:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493723776:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760739.213424:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23728128, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:1.0:1789760739.213433:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05491 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.213448:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760739.213453:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.213459:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05491 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.213473:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05491 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760739.246930:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973112 is committed 00002000:00010000:1.0:1789760739.247079:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23728128 -> 23732224 00010000:00010000:1.0:1789760739.247095:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05491 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.247108:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05491 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.247118:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05491 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.247131:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760739.247136:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05491 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.247146:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05491 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.247165:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05491 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.247172:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05491 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.247268:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493723776:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33901us (34124us total) trans 4294973112 rc 0/0 00000100:00100000:1.0:1789760739.247278:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5823 00000100:00100000:1.0:1789760739.247288:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749511863307 time 749554386054 state 0x2 ref 0 00000100:00100000:3.0:1789760739.252370:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760739.252551:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493723904 00000100:00100000:0.0:1789760739.252562:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493723904 00000100:00100000:0.0:1789760739.252615:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749511863307 time 749554386054 state 0x0 ref 1 00000100:00100000:0.0:1789760739.252630:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749554646470 time 749554386054 state 0x1 ref 1 00000100:00100000:0.0:1789760739.252662:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749554646470 time 749593697979 state 0x0 ref 1 00000100:00100000:0.0:1789760739.252675:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493723904:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760739.252722:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23732224, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:0.0:1789760739.252731:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05498 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.252745:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760739.252750:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.252755:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05498 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.252770:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05498 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760739.282592:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973113 is committed 00002000:00010000:1.0:1789760739.283691:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23732224 -> 23736320 00010000:00010000:1.0:1789760739.283708:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05498 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.283721:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05498 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.283735:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05498 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.283742:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760739.283747:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05498 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.283756:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05498 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.283776:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05498 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.283784:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05498 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.283881:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493723904:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31202us (31516us total) trans 4294973113 rc 0/0 00000100:00100000:1.0:1789760739.283892:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5824 00000100:00100000:1.0:1789760739.283901:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749554646470 time 749593697979 state 0x2 ref 0 00000100:00100000:0.0:1789760739.294060:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760739.294419:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493724032 00000100:00100000:2.0:1789760739.294429:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493724032 00000100:00100000:2.0:1789760739.294486:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749554646470 time 749593697979 state 0x0 ref 1 00000100:00100000:2.0:1789760739.294514:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749593958395 time 749593697979 state 0x1 ref 1 00000100:00100000:2.0:1789760739.294663:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749593958395 time 749635696963 state 0x0 ref 1 00000100:00100000:2.0:1789760739.294677:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493724032:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760739.294736:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23736320, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:2.0:1789760739.294746:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0549f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.294767:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760739.294772:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.294778:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0549f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.294793:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0549f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760739.354951:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973114 is committed 00002000:00010000:2.0:1789760739.355087:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23736320 -> 23740416 00010000:00010000:2.0:1789760739.355103:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0549f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.355118:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0549f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.355128:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0549f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.355136:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760739.355141:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0549f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.355151:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0549f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.355169:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0549f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.355179:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0549f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.355267:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493724032:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60588us (61213us total) trans 4294973114 rc 0/0 00000100:00100000:2.0:1789760739.355278:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5825 00000100:00100000:2.0:1789760739.355286:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749593958395 time 749635696963 state 0x2 ref 0 00000100:00100000:3.0:1789760739.370016:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760739.370104:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493724160 00000100:00100000:0.0:1789760739.370114:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493724160 00000100:00100000:0.0:1789760739.370173:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749593958395 time 749635696963 state 0x0 ref 1 00000100:00100000:0.0:1789760739.370188:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749635957379 time 749635696963 state 0x1 ref 1 00000100:00100000:0.0:1789760739.370218:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749635957379 time 749711254099 state 0x0 ref 1 00000100:00100000:0.0:1789760739.370230:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493724160:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760739.370276:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23740416, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:0.0:1789760739.370285:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e054a6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.370301:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:0.0:1789760739.370306:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.370312:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e054a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.370327:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e054a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760739.397596:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973115 is committed 00002000:00010000:1.0:1789760739.397773:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23740416 -> 23744512 00010000:00010000:1.0:1789760739.397789:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e054a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.397810:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e054a6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.397820:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e054a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.397827:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760739.397831:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e054a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.397841:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e054a6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.397860:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e054a6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.397867:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e054a6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.397957:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493724160:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27723us (27945us total) trans 4294973115 rc 0/0 00000100:00100000:1.0:1789760739.397998:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5826 00000100:00100000:1.0:1789760739.398008:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749635957379 time 749711254099 state 0x2 ref 0 00000100:00100000:0.0:1789760739.401667:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760739.401802:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493724288 00000100:00100000:1.0:1789760739.401815:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493724288 00000100:00100000:1.0:1789760739.401874:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749635957379 time 749711254099 state 0x0 ref 1 00000100:00100000:1.0:1789760739.401891:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749711514515 time 749711254099 state 0x1 ref 1 00000100:00100000:1.0:1789760739.402007:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749711514515 time 749743040976 state 0x0 ref 1 00000100:00100000:1.0:1789760739.402020:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493724288:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760739.402071:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23744512, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:1.0:1789760739.402080:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e054ad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.402095:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760739.402100:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.402107:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e054ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.402121:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e054ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760739.434799:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23744512 -> 23748608 00010000:00010000:0.0:1789760739.434821:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e054ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.434835:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e054ad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.434849:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e054ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.434857:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760739.434862:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e054ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.434871:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e054ad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.434892:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e054ad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.434910:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e054ad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760739.434982:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973116 is committed 00000100:00100000:0.0:1789760739.435047:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493724288:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33024us (33387us total) trans 4294973116 rc 0/0 00000100:00100000:0.0:1789760739.435058:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5827 00000100:00100000:0.0:1789760739.435068:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749711514515 time 749743040976 state 0x2 ref 0 00000100:00100000:3.0:1789760739.439154:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760739.439252:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493724416 00000100:00100000:1.0:1789760739.439263:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493724416 00000100:00100000:1.0:1789760739.439318:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749711514515 time 749743040976 state 0x0 ref 1 00000100:00100000:1.0:1789760739.439333:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749743301392 time 749743040976 state 0x1 ref 1 00000100:00100000:1.0:1789760739.439391:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749743301392 time 749780426883 state 0x0 ref 1 00000100:00100000:1.0:1789760739.439406:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493724416:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760739.439474:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23748608, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:1.0:1789760739.439483:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e054b4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.439511:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760739.439516:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.439522:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e054b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.439545:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e054b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760739.477003:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973117 is committed 00002000:00010000:0.0:1789760739.477130:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23748608 -> 23752704 00010000:00010000:0.0:1789760739.477148:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e054b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.477167:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e054b4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.477175:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e054b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.477182:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760739.477187:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e054b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.477197:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e054b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.477344:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e054b4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.477353:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e054b4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.477450:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493724416:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38040us (38300us total) trans 4294973117 rc 0/0 00000100:00100000:0.0:1789760739.477460:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5828 00000100:00100000:0.0:1789760739.477471:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749743301392 time 749780426883 state 0x2 ref 0 00000100:00100000:3.0:1789760739.484918:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760739.485346:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493724544 00000100:00100000:2.0:1789760739.485357:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493724544 00000100:00100000:2.0:1789760739.485419:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749743301392 time 749780426883 state 0x0 ref 1 00000100:00100000:2.0:1789760739.485434:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749780687299 time 749780426883 state 0x1 ref 1 00000100:00100000:2.0:1789760739.485468:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749780687299 time 749826503610 state 0x0 ref 1 00000100:00100000:2.0:1789760739.485482:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493724544:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760739.485543:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23752704, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:2.0:1789760739.485561:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e054bb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.485575:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760739.485579:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.485585:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e054bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.485599:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e054bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760739.524784:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973118 is committed 00002000:00010000:3.0:1789760739.524958:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23752704 -> 23756800 00010000:00010000:3.0:1789760739.525612:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e054bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.525630:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e054bb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.525639:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e054bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.525646:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760739.525651:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e054bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.525660:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e054bb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.525681:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e054bb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.525689:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e054bb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.525785:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493724544:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40301us (40883us total) trans 4294973118 rc 0/0 00000100:00100000:3.0:1789760739.525796:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5829 00000100:00100000:3.0:1789760739.525805:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749780687299 time 749826503610 state 0x2 ref 0 00000100:00100000:0.0:1789760739.529387:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760739.529461:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493724672 00000100:00100000:3.0:1789760739.529471:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493724672 00000100:00100000:3.0:1789760739.529531:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749780687299 time 749826503610 state 0x0 ref 1 00000100:00100000:3.0:1789760739.529556:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749826764026 time 749826503610 state 0x1 ref 1 00000100:00100000:3.0:1789760739.529620:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749826764026 time 749870655146 state 0x0 ref 1 00000100:00100000:3.0:1789760739.529633:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7aa7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493724672:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760739.529681:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23756800, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:3.0:1789760739.529690:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e054c2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.529704:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760739.529708:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.529715:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e054c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.529729:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e054c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760739.585699:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973119 is committed 00002000:00010000:3.0:1789760739.585848:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23756800 -> 23760896 00010000:00010000:3.0:1789760739.585863:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e054c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.585875:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e054c2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.585890:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e054c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.585897:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760739.585902:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e054c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.585911:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e054c2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.585932:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e054c2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.585939:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e054c2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.586443:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7aa7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493724672:12345-192.168.203.60@tcp:4:dd.0 Request processed in 56806us (57058us total) trans 4294973119 rc 0/0 00000100:00100000:3.0:1789760739.586452:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5830 00000100:00100000:3.0:1789760739.586462:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749826764026 time 749870655146 state 0x2 ref 0 00000100:00100000:0.0:1789760739.593363:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760739.593532:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493724800 00000100:00100000:1.0:1789760739.593543:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493724800 00000100:00100000:1.0:1789760739.593665:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749826764026 time 749870655146 state 0x0 ref 1 00000100:00100000:1.0:1789760739.593681:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749870915562 time 749870655146 state 0x1 ref 1 00000100:00100000:1.0:1789760739.593750:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749870915562 time 749934784820 state 0x0 ref 1 00000100:00100000:1.0:1789760739.593762:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493724800:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760739.593809:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23760896, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xb807 00010000:00010000:1.0:1789760739.593818:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e054c9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.593839:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760739.593843:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.593850:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e054c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.593864:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e054c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760739.626678:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23760896 -> 23764992 00002000:00010000:0.0:1789760739.626687:0:13038:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760738 -> 1789760739 00002000:00010000:0.0:1789760739.626690:0:13038:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760738 -> 1789760739 00010000:00010000:0.0:1789760739.626703:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e054c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.626717:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e054c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.626724:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e054c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.626731:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760739.626736:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e054c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.626745:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e054c9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.626765:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e054c9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.626772:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e054c9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.626903:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493724800:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33137us (33542us total) trans 4294973120 rc 0/0 00000100:00100000:0.0:1789760739.626914:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5831 00000100:00100000:0.0:1789760739.626923:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749870915562 time 749934784820 state 0x2 ref 0 00000001:00080000:0.0:1789760739.627004:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973120 is committed 00000100:00100000:0.0:1789760739.635609:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760739.635756:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493724928 00000100:00100000:1.0:1789760739.635766:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493724928 00000100:00100000:1.0:1789760739.635828:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749870915562 time 749934784820 state 0x0 ref 1 00000100:00100000:1.0:1789760739.635844:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749935045236 time 749934784820 state 0x1 ref 1 00000100:00100000:1.0:1789760739.635872:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749935045236 time 749976907803 state 0x0 ref 1 00000100:00100000:1.0:1789760739.635885:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493724928:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760739.635941:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23764992, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:1.0:1789760739.635951:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e054d0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.636003:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760739.636074:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.636082:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e054d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.636102:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e054d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760739.665144:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973121 is committed 00002000:00010000:2.0:1789760739.665259:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23764992 -> 23769088 00010000:00010000:2.0:1789760739.665276:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e054d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.665288:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e054d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.665298:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e054d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.665306:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760739.665310:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e054d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.665320:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e054d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.665341:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e054d0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.665349:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e054d0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.665499:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493724928:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29601us (29887us total) trans 4294973121 rc 0/0 00000100:00100000:2.0:1789760739.665510:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5832 00000100:00100000:2.0:1789760739.665519:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749935045236 time 749976907803 state 0x2 ref 0 00000100:00100000:3.0:1789760739.669896:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760739.672793:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493725056 00000100:00100000:2.0:1789760739.672809:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493725056 00000100:00100000:2.0:1789760739.672872:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749935045236 time 749976907803 state 0x0 ref 1 00000100:00100000:2.0:1789760739.672889:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749977168219 time 749976907803 state 0x1 ref 1 00000100:00100000:2.0:1789760739.673049:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749977168219 time 750014083676 state 0x0 ref 1 00000100:00100000:2.0:1789760739.673065:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8555180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493725056:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760739.673111:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23769088, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:2.0:1789760739.673120:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e054d7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.673137:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760739.673142:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.673148:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e054d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.673162:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e054d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760739.729359:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23769088 -> 23773184 00010000:00010000:0.0:1789760739.729379:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e054d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.729391:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e054d7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.729399:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e054d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.729406:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760739.729411:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e054d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.729420:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e054d7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.729441:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e054d7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.729450:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e054d7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.729569:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8555180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493725056:12345-192.168.203.60@tcp:4:dd.0 Request processed in 56502us (59675us total) trans 4294973122 rc 0/0 00000100:00100000:0.0:1789760739.729585:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5833 00000100:00100000:0.0:1789760739.729596:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749977168219 time 750014083676 state 0x2 ref 0 00000001:00080000:0.0:1789760739.729636:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973122 is committed 00000100:00100000:0.0:1789760739.737080:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760739.737200:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493725184 00000100:00100000:3.0:1789760739.737209:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493725184 00000100:00100000:3.0:1789760739.737261:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 749977168219 time 750014083676 state 0x0 ref 1 00000100:00100000:3.0:1789760739.737279:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750014344092 time 750014083676 state 0x1 ref 1 00000100:00100000:3.0:1789760739.737311:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750014344092 time 750078346561 state 0x0 ref 1 00000100:00100000:3.0:1789760739.737324:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493725184:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760739.737370:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23773184, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:3.0:1789760739.737379:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e054de lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.737394:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760739.737398:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.737405:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e054de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.737419:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e054de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760739.770709:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973123 is committed 00002000:00010000:3.0:1789760739.770821:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23773184 -> 23777280 00010000:00010000:3.0:1789760739.770836:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e054de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.770848:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e054de lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.770857:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e054de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.770869:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760739.770874:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e054de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.770882:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e054de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.770899:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e054de lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.770906:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e054de lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.771020:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493725184:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33694us (33946us total) trans 4294973123 rc 0/0 00000100:00100000:3.0:1789760739.771030:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5834 00000100:00100000:3.0:1789760739.771040:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750014344092 time 750078346561 state 0x2 ref 0 00000100:00100000:3.0:1789760739.773409:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760739.773520:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493725312 00000100:00100000:1.0:1789760739.773530:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493725312 00000100:00100000:1.0:1789760739.773586:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750014344092 time 750078346561 state 0x0 ref 1 00000100:00100000:1.0:1789760739.773600:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750078606977 time 750078346561 state 0x1 ref 1 00000100:00100000:1.0:1789760739.773638:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750078606977 time 750114673497 state 0x0 ref 1 00000100:00100000:1.0:1789760739.773650:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493725312:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760739.773699:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23777280, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:1.0:1789760739.773707:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e054e5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.773721:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760739.773726:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.773732:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e054e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.773745:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e054e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760739.815213:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973124 is committed 00002000:00010000:3.0:1789760739.815431:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23777280 -> 23781376 00010000:00010000:3.0:1789760739.815447:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e054e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.815460:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e054e5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.815486:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e054e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.815505:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760739.815511:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e054e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.815520:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e054e5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.815541:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e054e5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.815560:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e054e5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.815658:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493725312:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42005us (42253us total) trans 4294973124 rc 0/0 00000100:00100000:3.0:1789760739.815668:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5835 00000100:00100000:3.0:1789760739.815678:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750078606977 time 750114673497 state 0x2 ref 0 00000100:00100000:3.0:1789760739.820373:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760739.820469:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493725440 00000100:00100000:2.0:1789760739.820479:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493725440 00000100:00100000:2.0:1789760739.820557:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750078606977 time 750114673497 state 0x0 ref 1 00000100:00100000:2.0:1789760739.820574:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750114933913 time 750114673497 state 0x1 ref 1 00000100:00100000:2.0:1789760739.820604:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750114933913 time 750161640060 state 0x0 ref 1 00000100:00100000:2.0:1789760739.820617:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8556a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493725440:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760739.820667:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23781376, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:2.0:1789760739.820687:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e054ec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.820706:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760739.820711:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.820718:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e054ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.820732:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e054ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760739.856179:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23781376 -> 23785472 00010000:00010000:1.0:1789760739.856200:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e054ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.856212:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e054ec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.856221:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e054ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.856230:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760739.856235:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e054ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.856245:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e054ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.856265:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e054ec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.856273:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e054ec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.856361:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8556a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493725440:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35742us (35993us total) trans 4294973125 rc 0/0 00000100:00100000:1.0:1789760739.856371:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5836 00000100:00100000:1.0:1789760739.856381:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750114933913 time 750161640060 state 0x2 ref 0 00000001:00080000:3.0:1789760739.858419:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973125 is committed 00000100:00100000:0.0:1789760739.860364:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760739.860909:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493725568 00000100:00100000:2.0:1789760739.860919:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493725568 00000100:00100000:2.0:1789760739.861016:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750114933913 time 750161640060 state 0x0 ref 1 00000100:00100000:2.0:1789760739.861031:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750161900476 time 750161640060 state 0x1 ref 1 00000100:00100000:3.0:1789760739.861167:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750161900476 time 750202199532 state 0x0 ref 1 00000100:00100000:3.0:1789760739.861185:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493725568:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760739.861233:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23785472, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:3.0:1789760739.861242:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e054f3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.861257:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760739.861261:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.861267:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e054f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.861279:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e054f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760739.892456:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23785472 -> 23789568 00010000:00010000:3.0:1789760739.892474:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e054f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.892488:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e054f3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.892506:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e054f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.892514:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760739.892518:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e054f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.892527:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e054f3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.892554:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e054f3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.892562:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e054f3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.892652:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493725568:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31466us (32294us total) trans 4294973126 rc 0/0 00000100:00100000:3.0:1789760739.892662:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5837 00000100:00100000:3.0:1789760739.892672:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750161900476 time 750202199532 state 0x2 ref 0 00000001:00080000:1.0:1789760739.894112:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973126 is committed 00010000:00010000:2.0:1789760739.898165:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760739.901521:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760739.901710:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493725696 00000100:00100000:3.0:1789760739.901822:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493725696 00000100:00100000:3.0:1789760739.901880:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750161900476 time 750202199532 state 0x0 ref 1 00000100:00100000:3.0:1789760739.901895:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750202459948 time 750202199532 state 0x1 ref 1 00000100:00100000:3.0:1789760739.901930:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750202459948 time 750242965947 state 0x0 ref 1 00000100:00100000:3.0:1789760739.901943:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493725696:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760739.902037:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23789568, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:3.0:1789760739.902047:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e054fa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.902062:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760739.902067:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.902073:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e054fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.902085:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e054fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760739.963509:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973127 is committed 00002000:00010000:3.0:1789760739.963604:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23789568 -> 23793664 00010000:00010000:3.0:1789760739.963623:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e054fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.963636:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e054fa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.963645:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e054fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.963652:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760739.963657:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e054fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.963667:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e054fa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.963683:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e054fa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.963691:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e054fa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.963772:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493725696:12345-192.168.203.60@tcp:4:dd.0 Request processed in 61827us (62257us total) trans 4294973127 rc 0/0 00000100:00100000:3.0:1789760739.963782:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5838 00000100:00100000:3.0:1789760739.963792:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750202459948 time 750242965947 state 0x2 ref 0 00000100:00100000:2.0:1789760739.972057:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760739.972332:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493725824 00000100:00100000:0.0:1789760739.972342:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493725824 00000100:00100000:0.0:1789760739.972397:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750202459948 time 750242965947 state 0x0 ref 1 00000100:00100000:0.0:1789760739.972412:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750243226363 time 750242965947 state 0x1 ref 1 00000100:00100000:0.0:1789760739.972443:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750243226363 time 750313477963 state 0x0 ref 1 00000100:00100000:0.0:1789760739.972456:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493725824:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760739.972513:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23793664, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:0.0:1789760739.972523:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05501 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.972539:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760739.972543:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.972559:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05501 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.972574:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05501 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.011903:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973128 is committed 00002000:00010000:3.0:1789760740.012171:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23793664 -> 23797760 00010000:00010000:3.0:1789760740.012185:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05501 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.012198:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05501 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.012207:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05501 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.012214:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760740.012218:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05501 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.012228:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05501 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.012251:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05501 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.012258:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05501 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.012353:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493725824:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39894us (40297us total) trans 4294973128 rc 0/0 00000100:00100000:3.0:1789760740.012373:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5839 00000100:00100000:3.0:1789760740.012382:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750243226363 time 750313477963 state 0x2 ref 0 00000100:00100000:2.0:1789760740.022101:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760740.022260:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493725952 00000100:00100000:1.0:1789760740.022270:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493725952 00000100:00100000:1.0:1789760740.022321:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750243226363 time 750313477963 state 0x0 ref 1 00000100:00100000:1.0:1789760740.022336:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750313738379 time 750313477963 state 0x1 ref 1 00000100:00100000:1.0:1789760740.022412:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750313738379 time 750363447567 state 0x0 ref 1 00000100:00100000:1.0:1789760740.022425:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb04450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493725952:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760740.022471:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23797760, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:1.0:1789760740.022479:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05508 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.022506:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760740.022510:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760740.022516:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05508 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.022530:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05508 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.076387:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973129 is committed 00002000:00010000:3.0:1789760740.076611:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23797760 -> 23801856 00010000:00010000:3.0:1789760740.076627:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05508 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.076641:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05508 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.076648:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05508 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.076656:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760740.076661:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05508 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.076674:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05508 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.076694:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05508 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.076702:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05508 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.076798:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb04450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493725952:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54371us (54701us total) trans 4294973129 rc 0/0 00000100:00100000:3.0:1789760740.076810:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5840 00000100:00100000:3.0:1789760740.076821:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750313738379 time 750363447567 state 0x2 ref 0 00000100:00100000:2.0:1789760740.084221:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760740.084312:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493726080 00000100:00100000:1.0:1789760740.084324:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493726080 00000100:00100000:1.0:1789760740.084387:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750313738379 time 750363447567 state 0x0 ref 1 00000100:00100000:1.0:1789760740.084404:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750363707983 time 750363447567 state 0x1 ref 1 00000100:00100000:1.0:1789760740.084500:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750363707983 time 750425526107 state 0x0 ref 1 00000100:00100000:1.0:1789760740.084515:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493726080:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760740.084576:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23801856, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:1.0:1789760740.084585:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0550f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.084611:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760740.084615:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760740.084621:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0550f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.084636:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0550f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760740.138388:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973130 is committed 00002000:00010000:3.0:1789760740.138633:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23801856 -> 23805952 00010000:00010000:3.0:1789760740.138647:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0550f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.138660:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0550f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.138668:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0550f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.138674:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760740.138679:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0550f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.138688:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0550f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.138706:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0550f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.138713:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0550f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.138837:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493726080:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54320us (54624us total) trans 4294973130 rc 0/0 00000100:00100000:3.0:1789760740.138847:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5841 00000100:00100000:3.0:1789760740.138857:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750363707983 time 750425526107 state 0x2 ref 0 00000100:00100000:3.0:1789760740.154228:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760740.154265:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493726208 00000100:00100000:3.0:1789760740.154275:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493726208 00000100:00100000:3.0:1789760740.154333:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750363707983 time 750425526107 state 0x0 ref 1 00000100:00100000:3.0:1789760740.154349:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750425786523 time 750425526107 state 0x1 ref 1 00000100:00100000:3.0:1789760740.154375:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750425786523 time 750495410434 state 0x0 ref 1 00000100:00100000:3.0:1789760740.154387:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8555180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493726208:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760740.154440:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23805952, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:3.0:1789760740.154449:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05516 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.154465:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760740.154469:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.154476:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05516 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.154502:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05516 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.194420:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973131 is committed 00002000:00010000:3.0:1789760740.194551:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23805952 -> 23810048 00010000:00010000:3.0:1789760740.194567:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05516 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.194580:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05516 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.194589:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05516 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.194597:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760740.194602:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05516 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.194612:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05516 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.194633:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05516 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.194643:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05516 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.194739:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8555180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493726208:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40348us (40516us total) trans 4294973131 rc 0/0 00000100:00100000:3.0:1789760740.194751:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5842 00000100:00100000:3.0:1789760740.194760:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750425786523 time 750495410434 state 0x2 ref 0 00000100:00100000:2.0:1789760740.198893:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760740.199006:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493726336 00000100:00100000:0.0:1789760740.199016:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493726336 00000100:00100000:0.0:1789760740.199069:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750425786523 time 750495410434 state 0x0 ref 1 00000100:00100000:0.0:1789760740.199083:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750495670850 time 750495410434 state 0x1 ref 1 00000100:00100000:0.0:1789760740.199108:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750495670850 time 750540144150 state 0x0 ref 1 00000100:00100000:0.0:1789760740.199120:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493726336:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760740.199166:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23810048, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:0.0:1789760740.199176:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0551d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.199193:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760740.199197:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.199203:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0551d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.199218:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0551d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.231513:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973132 is committed 00002000:00010000:3.0:1789760740.231661:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23810048 -> 23814144 00010000:00010000:3.0:1789760740.231680:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0551d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.231695:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0551d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.231705:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0551d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.231713:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760740.231717:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0551d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.231733:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0551d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.231757:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0551d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.231767:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0551d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.231877:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493726336:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32755us (32988us total) trans 4294973132 rc 0/0 00000100:00100000:3.0:1789760740.231888:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5843 00000100:00100000:3.0:1789760740.231898:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750495670850 time 750540144150 state 0x2 ref 0 00000100:00100000:2.0:1789760740.245860:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760740.245955:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493726464 00000100:00100000:3.0:1789760740.246055:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493726464 00000100:00100000:2.0:1789760740.246450:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750495670850 time 750540144150 state 0x0 ref 1 00000100:00100000:2.0:1789760740.246467:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750540404566 time 750540144150 state 0x1 ref 1 00000100:00100000:2.0:1789760740.246703:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750540404566 time 750587736689 state 0x0 ref 1 00000100:00100000:2.0:1789760740.246718:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493726464:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760740.246766:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23814144, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:2.0:1789760740.246776:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05524 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.246793:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760740.246798:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.246805:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05524 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.246826:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05524 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760740.269021:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23814144 -> 23818240 00010000:00010000:3.0:1789760740.269048:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05524 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.269063:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05524 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.269072:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05524 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.269085:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760740.269091:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05524 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.269101:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05524 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.269120:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05524 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.269128:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05524 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.269137:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973133 is committed 00000100:00100000:3.0:1789760740.269216:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493726464:12345-192.168.203.60@tcp:4:dd.0 Request processed in 22497us (23361us total) trans 4294973133 rc 0/0 00000100:00100000:3.0:1789760740.269227:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5844 00000100:00100000:3.0:1789760740.269236:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750540404566 time 750587736689 state 0x2 ref 0 00000100:00100000:2.0:1789760740.272170:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760740.272268:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493726592 00000100:00100000:3.0:1789760740.272277:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493726592 00000100:00100000:3.0:1789760740.272567:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750540404566 time 750587736689 state 0x0 ref 1 00000100:00100000:3.0:1789760740.272583:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750587997105 time 750587736689 state 0x1 ref 1 00000100:00100000:3.0:1789760740.272664:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750587997105 time 750613698785 state 0x0 ref 1 00000100:00100000:3.0:1789760740.272682:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493726592:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760740.272742:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23818240, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:3.0:1789760740.272751:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0552b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.272770:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760740.272775:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.272781:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0552b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.272795:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0552b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.305793:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973134 is committed 00002000:00010000:3.0:1789760740.305878:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23818240 -> 23822336 00010000:00010000:3.0:1789760740.305893:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0552b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.305905:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0552b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.305914:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0552b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.305921:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760740.305926:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0552b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.305936:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0552b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.305956:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0552b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.306055:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0552b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.306235:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493726592:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33551us (34070us total) trans 4294973134 rc 0/0 00000100:00100000:2.0:1789760740.306247:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5845 00000100:00100000:2.0:1789760740.306257:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750587997105 time 750613698785 state 0x2 ref 0 00000100:00100000:2.0:1789760740.313512:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760740.313798:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493726720 00000100:00100000:3.0:1789760740.313808:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493726720 00000100:00100000:3.0:1789760740.313862:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750587997105 time 750613698785 state 0x0 ref 1 00000100:00100000:3.0:1789760740.313877:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750613959201 time 750613698785 state 0x1 ref 1 00000100:00100000:3.0:1789760740.314099:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750613959201 time 750654947955 state 0x0 ref 1 00000100:00100000:3.0:1789760740.314116:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493726720:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760740.314164:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23822336, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:3.0:1789760740.314173:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05532 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.314187:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760740.314192:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.314199:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05532 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.314213:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05532 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760740.352263:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973135 is committed 00002000:00010000:3.0:1789760740.352418:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23822336 -> 23826432 00010000:00010000:3.0:1789760740.352433:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05532 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.352446:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05532 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.352464:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05532 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.352472:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760740.352477:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05532 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.352485:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05532 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.352514:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05532 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.352522:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05532 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.352735:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493726720:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38616us (39307us total) trans 4294973135 rc 0/0 00000100:00100000:3.0:1789760740.352747:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5846 00000100:00100000:3.0:1789760740.352759:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750613959201 time 750654947955 state 0x2 ref 0 00000100:00100000:0.0:1789760740.365768:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760740.365944:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493726848 00000100:00100000:3.0:1789760740.365955:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493726848 00000100:00100000:3.0:1789760740.366023:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750613959201 time 750654947955 state 0x0 ref 1 00000100:00100000:3.0:1789760740.366038:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750655208371 time 750654947955 state 0x1 ref 1 00000100:00100000:3.0:1789760740.366066:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750655208371 time 750707101287 state 0x0 ref 1 00000100:00100000:3.0:1789760740.366079:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493726848:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760740.366127:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23826432, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:3.0:1789760740.366135:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05539 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.366152:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760740.366156:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.366162:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05539 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.366175:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05539 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.416353:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973136 is committed 00002000:00010000:3.0:1789760740.416665:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23826432 -> 23830528 00010000:00010000:3.0:1789760740.416680:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05539 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.416692:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05539 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.416701:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05539 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.416708:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760740.416713:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05539 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.416734:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05539 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.416753:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05539 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.416760:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05539 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.416853:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493726848:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50772us (51090us total) trans 4294973136 rc 0/0 00000100:00100000:3.0:1789760740.417004:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5847 00000100:00100000:3.0:1789760740.417016:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750655208371 time 750707101287 state 0x2 ref 0 00000100:00100000:2.0:1789760740.420854:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760740.421174:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493726976 00000100:00100000:3.0:1789760740.421185:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493726976 00000100:00100000:0.0:1789760740.421374:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750655208371 time 750707101287 state 0x0 ref 1 00000100:00100000:0.0:1789760740.421391:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750707361703 time 750707101287 state 0x1 ref 1 00000100:00100000:0.0:1789760740.421465:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750707361703 time 750762499702 state 0x0 ref 1 00000100:00100000:0.0:1789760740.421479:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493726976:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760740.421541:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23830528, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:0.0:1789760740.421550:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05540 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.421566:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760740.421570:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.421576:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05540 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.421590:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05540 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.457876:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973137 is committed 00002000:00010000:3.0:1789760740.458413:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23830528 -> 23834624 00010000:00010000:3.0:1789760740.458428:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05540 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.458441:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05540 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.458451:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05540 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.458458:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760740.458462:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05540 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.458472:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05540 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.458502:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05540 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.458511:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05540 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.458649:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493726976:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37168us (37801us total) trans 4294973137 rc 0/0 00000100:00100000:3.0:1789760740.458659:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5848 00000100:00100000:3.0:1789760740.458668:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750707361703 time 750762499702 state 0x2 ref 0 00000100:00100000:2.0:1789760740.464307:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760740.464390:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493727104 00000100:00100000:1.0:1789760740.464400:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493727104 00000100:00100000:1.0:1789760740.464474:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750707361703 time 750762499702 state 0x0 ref 1 00000100:00100000:1.0:1789760740.464499:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750762760118 time 750762499702 state 0x1 ref 1 00000100:00100000:1.0:1789760740.464529:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750762760118 time 750805565001 state 0x0 ref 1 00000100:00100000:1.0:1789760740.464542:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493727104:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760740.464698:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23834624, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:1.0:1789760740.464708:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05547 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.464724:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760740.464729:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760740.464734:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05547 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.464747:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05547 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.507753:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973138 is committed 00002000:00010000:3.0:1789760740.507861:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23834624 -> 23838720 00010000:00010000:3.0:1789760740.507876:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05547 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.507897:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05547 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.507907:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05547 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.507913:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760740.507918:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05547 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.507928:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05547 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.507947:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05547 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.507994:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05547 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.508074:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493727104:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43529us (43773us total) trans 4294973138 rc 0/0 00000100:00100000:3.0:1789760740.508083:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5849 00000100:00100000:3.0:1789760740.508092:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750762760118 time 750805565001 state 0x2 ref 0 00000100:00100000:2.0:1789760740.511229:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760740.511358:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493727232 00000100:00100000:3.0:1789760740.511369:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493727232 00000100:00100000:3.0:1789760740.511423:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750762760118 time 750805565001 state 0x0 ref 1 00000100:00100000:3.0:1789760740.511438:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750805825417 time 750805565001 state 0x1 ref 1 00000100:00100000:3.0:1789760740.511474:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750805825417 time 750852510111 state 0x0 ref 1 00000100:00100000:3.0:1789760740.511496:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493727232:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760740.511556:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23838720, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:3.0:1789760740.511565:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0554e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.511579:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760740.511584:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.511590:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0554e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.511609:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0554e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760740.577320:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23838720 -> 23842816 00010000:00010000:3.0:1789760740.577343:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0554e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.577348:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973139 is committed 00010000:00010000:3.0:1789760740.577359:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0554e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.577370:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0554e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.577378:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760740.577384:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0554e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.577393:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0554e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.577417:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0554e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.577427:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0554e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.577554:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493727232:12345-192.168.203.60@tcp:4:dd.0 Request processed in 66063us (66326us total) trans 4294973139 rc 0/0 00000100:00100000:3.0:1789760740.577564:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5850 00000100:00100000:3.0:1789760740.577646:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750805825417 time 750852510111 state 0x2 ref 0 00000100:00100000:2.0:1789760740.581264:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760740.581649:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493727360 00000100:00100000:3.0:1789760740.581660:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493727360 00000100:00100000:3.0:1789760740.581715:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750805825417 time 750852510111 state 0x0 ref 1 00000100:00100000:3.0:1789760740.581730:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750852770527 time 750852510111 state 0x1 ref 1 00000100:00100000:3.0:1789760740.581821:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750852770527 time 750922855557 state 0x0 ref 1 00000100:00100000:3.0:1789760740.581835:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493727360:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760740.581885:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23842816, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xb807 00010000:00010000:3.0:1789760740.581894:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05555 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.581911:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:3.0:1789760740.581916:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.581922:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05555 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.581937:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05555 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.616921:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973140 is committed 00002000:00010000:0.0:1789760740.616926:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23842816 -> 23846912 00002000:00010000:0.0:1789760740.616931:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760739 -> 1789760740 00002000:00010000:0.0:1789760740.616935:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760739 -> 1789760740 00010000:00010000:0.0:1789760740.616949:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05555 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.616993:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05555 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.617002:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05555 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.617009:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760740.617014:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05555 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.617024:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05555 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.617043:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05555 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.617050:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05555 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.617332:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493727360:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35494us (36073us total) trans 4294973140 rc 0/0 00000100:00100000:2.0:1789760740.617344:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5851 00000100:00100000:2.0:1789760740.617354:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750852770527 time 750922855557 state 0x2 ref 0 00000100:00100000:2.0:1789760740.621113:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760740.621193:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493727488 00000100:00100000:1.0:1789760740.621202:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493727488 00000100:00100000:1.0:1789760740.621258:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750852770527 time 750922855557 state 0x0 ref 1 00000100:00100000:1.0:1789760740.621274:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750923115973 time 750922855557 state 0x1 ref 1 00000100:00100000:1.0:1789760740.621304:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750923115973 time 750962339666 state 0x0 ref 1 00000100:00100000:1.0:1789760740.621317:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493727488:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760740.621364:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23846912, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:1.0:1789760740.621374:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0555c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.621389:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760740.621395:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760740.621401:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0555c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.621413:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0555c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760740.670115:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23846912 -> 23851008 00010000:00010000:1.0:1789760740.670136:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0555c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.670148:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0555c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.670156:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0555c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.670163:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760740.670167:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0555c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.670322:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973141 is committed 00010000:00010000:1.0:1789760740.670364:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0555c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.670391:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0555c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.670399:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0555c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760740.670658:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493727488:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49338us (49550us total) trans 4294973141 rc 0/0 00000100:00100000:1.0:1789760740.670670:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5852 00000100:00100000:1.0:1789760740.670679:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750923115973 time 750962339666 state 0x2 ref 0 00000100:00100000:0.0:1789760740.680208:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760740.680360:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493727616 00000100:00100000:1.0:1789760740.680370:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493727616 00000100:00100000:1.0:1789760740.680430:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750923115973 time 750962339666 state 0x0 ref 1 00000100:00100000:1.0:1789760740.680447:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750962600082 time 750962339666 state 0x1 ref 1 00000100:00100000:1.0:1789760740.680480:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750962600082 time 751021515903 state 0x0 ref 1 00000100:00100000:1.0:1789760740.680506:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc17480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493727616:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760740.680566:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23851008, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:1.0:1789760740.680575:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05563 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.680591:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760740.680596:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760740.680602:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05563 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.680615:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05563 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760740.725988:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973142 is committed 00002000:00010000:3.0:1789760740.726308:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23851008 -> 23855104 00010000:00010000:3.0:1789760740.726323:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05563 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.726337:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05563 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.726346:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05563 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.726352:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760740.726357:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05563 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.726365:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05563 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.726384:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05563 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.726391:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05563 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.726496:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc17480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493727616:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45979us (46294us total) trans 4294973142 rc 0/0 00000100:00100000:3.0:1789760740.726510:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5853 00000100:00100000:3.0:1789760740.726519:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750962600082 time 751021515903 state 0x2 ref 0 00000100:00100000:2.0:1789760740.733125:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760740.733698:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493727744 00000100:00100000:0.0:1789760740.733708:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493727744 00000100:00100000:0.0:1789760740.733762:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 750962600082 time 751021515903 state 0x0 ref 1 00000100:00100000:0.0:1789760740.733779:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751021776319 time 751021515903 state 0x1 ref 1 00000100:00100000:0.0:1789760740.733809:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751021776319 time 751074845192 state 0x0 ref 1 00000100:00100000:0.0:1789760740.733822:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493727744:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760740.733883:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23855104, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:0.0:1789760740.733892:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0556a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.733906:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:0.0:1789760740.733911:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.733917:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0556a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.733931:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0556a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.769292:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973143 is committed 00002000:00010000:3.0:1789760740.769460:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23855104 -> 23859200 00010000:00010000:3.0:1789760740.769475:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0556a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.769498:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0556a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.769506:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0556a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.769513:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760740.769518:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0556a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.769528:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0556a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.769556:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0556a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.769564:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0556a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.769665:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493727744:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35839us (36544us total) trans 4294973143 rc 0/0 00000100:00100000:3.0:1789760740.769676:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5854 00000100:00100000:3.0:1789760740.769685:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751021776319 time 751074845192 state 0x2 ref 0 00000100:00100000:2.0:1789760740.775924:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760740.776026:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493727872 00000100:00100000:3.0:1789760740.776036:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493727872 00000100:00100000:3.0:1789760740.776094:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751021776319 time 751074845192 state 0x0 ref 1 00000100:00100000:3.0:1789760740.776116:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751075105608 time 751074845192 state 0x1 ref 1 00000100:00100000:3.0:1789760740.776144:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751075105608 time 751117179592 state 0x0 ref 1 00000100:00100000:3.0:1789760740.776155:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493727872:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760740.776208:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23859200, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:3.0:1789760740.776218:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05571 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.776234:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760740.776238:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.776245:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05571 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.776258:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05571 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760740.820783:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973144 is committed 00002000:00010000:3.0:1789760740.820868:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23859200 -> 23863296 00010000:00010000:3.0:1789760740.820955:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05571 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.821043:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05571 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.821261:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05571 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.821269:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760740.821273:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05571 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.821292:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05571 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.821312:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05571 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.821320:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05571 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.821526:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493727872:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45366us (45604us total) trans 4294973144 rc 0/0 00000100:00100000:2.0:1789760740.821546:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5855 00000100:00100000:2.0:1789760740.821557:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751075105608 time 751117179592 state 0x2 ref 0 00000100:00100000:0.0:1789760740.826588:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760740.827674:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493728000 00000100:00100000:1.0:1789760740.827686:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493728000 00000100:00100000:0.0:1789760740.827836:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751075105608 time 751117179592 state 0x0 ref 1 00000100:00100000:0.0:1789760740.827852:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751117440008 time 751117179592 state 0x1 ref 1 00000100:00100000:0.0:1789760740.827994:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751117440008 time 751169027939 state 0x0 ref 1 00000100:00100000:0.0:1789760740.828008:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493728000:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760740.828061:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23863296, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:0.0:1789760740.828070:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05578 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.828087:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:0.0:1789760740.828092:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.828098:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05578 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.828112:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05578 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760740.868364:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973145 is committed 00002000:00010000:3.0:1789760740.868531:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23863296 -> 23867392 00010000:00010000:3.0:1789760740.868604:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05578 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.868616:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05578 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.868625:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05578 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.868632:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760740.868637:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05578 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.868646:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05578 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.868665:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05578 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.868673:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05578 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.868801:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493728000:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40790us (42217us total) trans 4294973145 rc 0/0 00000100:00100000:3.0:1789760740.868812:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5856 00000100:00100000:3.0:1789760740.868822:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751117440008 time 751169027939 state 0x2 ref 0 00000100:00100000:3.0:1789760740.874147:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760740.874463:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493728128 00000100:00100000:1.0:1789760740.874490:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493728128 00000100:00100000:1.0:1789760740.874554:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751117440008 time 751169027939 state 0x0 ref 1 00000100:00100000:1.0:1789760740.874571:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751169288355 time 751169027939 state 0x1 ref 1 00000100:00100000:1.0:1789760740.874603:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751169288355 time 751215638996 state 0x0 ref 1 00000100:00100000:1.0:1789760740.874616:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493728128:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760740.874661:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23867392, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:1.0:1789760740.874671:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0557f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.874691:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760740.874696:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760740.874702:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0557f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.874716:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0557f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760740.904840:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973146 is committed 00002000:00010000:3.0:1789760740.904874:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23867392 -> 23871488 00010000:00010000:3.0:1789760740.904888:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0557f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.904902:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0557f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.904911:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0557f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.904918:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760740.904923:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0557f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.904999:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0557f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.905021:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0557f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.905028:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0557f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.905172:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493728128:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30549us (31026us total) trans 4294973146 rc 0/0 00000100:00100000:3.0:1789760740.905183:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5857 00000100:00100000:3.0:1789760740.905192:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751169288355 time 751215638996 state 0x2 ref 0 00000100:00100000:2.0:1789760740.910619:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760740.910718:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493728256 00000100:00100000:3.0:1789760740.910728:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493728256 00000100:00100000:3.0:1789760740.910776:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751169288355 time 751215638996 state 0x0 ref 1 00000100:00100000:3.0:1789760740.910792:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751215899412 time 751215638996 state 0x1 ref 1 00000100:00100000:3.0:1789760740.910822:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751215899412 time 751251857684 state 0x0 ref 1 00000100:00100000:3.0:1789760740.910834:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493728256:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760740.910877:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23871488, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:3.0:1789760740.910885:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05586 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.910901:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760740.910906:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.910912:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05586 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.910925:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05586 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.922945:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760740.950997:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23871488 -> 23875584 00010000:00010000:3.0:1789760740.951016:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05586 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.951029:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05586 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760740.951034:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973147 is committed 00010000:00010000:3.0:1789760740.951039:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05586 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.951048:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760740.951054:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05586 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.951063:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05586 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.951084:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05586 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.951091:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05586 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760740.951215:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493728256:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40376us (40598us total) trans 4294973147 rc 0/0 00000100:00100000:3.0:1789760740.951225:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5858 00000100:00100000:3.0:1789760740.951235:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751215899412 time 751251857684 state 0x2 ref 0 00000100:00100000:0.0:1789760740.955840:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760740.955950:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493728384 00000100:00100000:2.0:1789760740.955990:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493728384 00000100:00100000:2.0:1789760740.956049:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751215899412 time 751251857684 state 0x0 ref 1 00000100:00100000:2.0:1789760740.956064:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751252118100 time 751251857684 state 0x1 ref 1 00000100:00100000:2.0:1789760740.956088:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751252118100 time 751297123678 state 0x0 ref 1 00000100:00100000:2.0:1789760740.956101:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493728384:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760740.956150:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23875584, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:2.0:1789760740.956159:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0558d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.956175:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760740.956179:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.956186:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0558d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.956201:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0558d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760741.017869:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973148 is committed 00002000:00010000:2.0:1789760741.018052:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23875584 -> 23879680 00010000:00010000:2.0:1789760741.018068:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0558d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.018083:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0558d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.018092:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0558d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.018100:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760741.018105:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0558d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.018128:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0558d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.018304:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0558d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.018314:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0558d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.018419:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493728384:12345-192.168.203.60@tcp:4:dd.0 Request processed in 62315us (62582us total) trans 4294973148 rc 0/0 00000100:00100000:2.0:1789760741.018430:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5859 00000100:00100000:2.0:1789760741.018452:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751252118100 time 751297123678 state 0x2 ref 0 00000100:00100000:2.0:1789760741.022094:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760741.022174:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493728512 00000100:00100000:0.0:1789760741.022184:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493728512 00000100:00100000:0.0:1789760741.022291:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751252118100 time 751297123678 state 0x0 ref 1 00000100:00100000:0.0:1789760741.022306:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751297384094 time 751297123678 state 0x1 ref 1 00000100:00100000:0.0:1789760741.022344:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751297384094 time 751363379393 state 0x0 ref 1 00000100:00100000:0.0:1789760741.022356:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493728512:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760741.022414:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23879680, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:0.0:1789760741.022426:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05594 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.022442:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:0.0:1789760741.022446:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.022453:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05594 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.022466:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05594 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760741.024028:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789760741.056168:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23879680 -> 23883776 00010000:00010000:3.0:1789760741.056188:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05594 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.056201:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05594 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.056210:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05594 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.056218:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760741.056223:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05594 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.056243:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05594 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.056261:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05594 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.056268:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05594 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.056358:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493728512:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33999us (34269us total) trans 4294973149 rc 0/0 00000100:00100000:3.0:1789760741.056367:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5860 00000100:00100000:3.0:1789760741.056377:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751297384094 time 751363379393 state 0x2 ref 0 00000001:00080000:1.0:1789760741.057587:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973149 is committed 00000100:00100000:0.0:1789760741.060628:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760741.060742:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493728640 00000100:00100000:3.0:1789760741.060752:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493728640 00000100:00100000:3.0:1789760741.060804:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751297384094 time 751363379393 state 0x0 ref 1 00000100:00100000:3.0:1789760741.060819:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751363639809 time 751363379393 state 0x1 ref 1 00000100:00100000:3.0:1789760741.060910:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751363639809 time 751401945199 state 0x0 ref 1 00000100:00100000:3.0:1789760741.060924:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493728640:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760741.061027:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23883776, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:3.0:1789760741.061036:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0559b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.061052:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760741.061056:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.061062:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0559b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.061076:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0559b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.098831:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973150 is committed 00002000:00010000:3.0:1789760741.099065:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23883776 -> 23887872 00010000:00010000:3.0:1789760741.099080:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0559b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.099091:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0559b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.099104:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0559b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.099111:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760741.099116:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0559b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.099124:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0559b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.099142:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0559b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.099149:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0559b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.099231:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493728640:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38305us (38609us total) trans 4294973150 rc 0/0 00000100:00100000:3.0:1789760741.099241:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5861 00000100:00100000:3.0:1789760741.099251:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751363639809 time 751401945199 state 0x2 ref 0 00000100:00100000:2.0:1789760741.102452:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760741.102630:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493728768 00000100:00100000:1.0:1789760741.102639:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493728768 00000100:00100000:1.0:1789760741.102691:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751363639809 time 751401945199 state 0x0 ref 1 00000100:00100000:1.0:1789760741.102705:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751402205615 time 751401945199 state 0x1 ref 1 00000100:00100000:1.0:1789760741.102766:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751402205615 time 751443801036 state 0x0 ref 1 00000100:00100000:1.0:1789760741.102780:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493728768:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760741.102827:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23887872, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:1.0:1789760741.102835:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e055a2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.102850:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760741.102855:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760741.102860:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e055a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.102873:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e055a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760741.133530:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23887872 -> 23891968 00010000:00010000:1.0:1789760741.133560:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e055a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.133569:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e055a2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.133576:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e055a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.133580:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760741.133584:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e055a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.133589:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e055a2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.133604:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e055a2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.133609:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e055a2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760741.133689:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493728768:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30908us (31242us total) trans 4294973151 rc 0/0 00000100:00100000:1.0:1789760741.133699:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5862 00000100:00100000:1.0:1789760741.133709:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751402205615 time 751443801036 state 0x2 ref 0 00000001:00080000:3.0:1789760741.134054:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973151 is committed 00000100:00100000:0.0:1789760741.137660:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760741.139666:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493728896 00000100:00100000:2.0:1789760741.139683:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493728896 00000100:00100000:2.0:1789760741.139756:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751402205615 time 751443801036 state 0x0 ref 1 00000100:00100000:2.0:1789760741.139773:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751444061452 time 751443801036 state 0x1 ref 1 00000100:00100000:2.0:1789760741.139810:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751444061452 time 751480845519 state 0x0 ref 1 00000100:00100000:2.0:1789760741.139824:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493728896:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760741.140204:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23891968, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:2.0:1789760741.140213:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e055a9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.140228:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760741.140232:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.140248:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e055a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.140269:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e055a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.170473:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973152 is committed 00002000:00010000:2.0:1789760741.170569:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23891968 -> 23896064 00010000:00010000:2.0:1789760741.170583:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e055a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.170595:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e055a9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.170602:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e055a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.170608:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760741.170613:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e055a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.170620:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e055a9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.170638:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e055a9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.170644:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e055a9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.170735:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493728896:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30910us (33080us total) trans 4294973152 rc 0/0 00000100:00100000:2.0:1789760741.170745:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5863 00000100:00100000:2.0:1789760741.170755:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751444061452 time 751480845519 state 0x2 ref 0 00000100:00100000:2.0:1789760741.173776:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760741.173913:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493729024 00000100:00100000:3.0:1789760741.173922:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493729024 00000100:00100000:3.0:1789760741.174056:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751444061452 time 751480845519 state 0x0 ref 1 00000100:00100000:3.0:1789760741.174074:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751481105935 time 751480845519 state 0x1 ref 1 00000100:00100000:3.0:1789760741.174367:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751481105935 time 751515400457 state 0x0 ref 1 00000100:00100000:3.0:1789760741.174380:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493729024:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760741.174429:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23896064, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:3.0:1789760741.174438:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055b0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.174453:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760741.174457:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.174464:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.174476:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760741.212493:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23896064 -> 23900160 00010000:00010000:3.0:1789760741.212514:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.212531:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055b0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.212538:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.212559:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760741.212564:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.212573:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055b0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.212592:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055b0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.212599:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055b0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.212691:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493729024:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38305us (38916us total) trans 4294973153 rc 0/0 00000100:00100000:3.0:1789760741.212702:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5864 00000100:00100000:3.0:1789760741.212711:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751481105935 time 751515400457 state 0x2 ref 0 00000001:00080000:2.0:1789760741.212712:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973153 is committed 00000100:00100000:0.0:1789760741.217516:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760741.217594:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493729152 00000100:00100000:3.0:1789760741.217604:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493729152 00000100:00100000:3.0:1789760741.217658:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751481105935 time 751515400457 state 0x0 ref 1 00000100:00100000:3.0:1789760741.217673:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751515660873 time 751515400457 state 0x1 ref 1 00000100:00100000:3.0:1789760741.217702:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751515660873 time 751558737925 state 0x0 ref 1 00000100:00100000:3.0:1789760741.217718:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493729152:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760741.217767:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23900160, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:3.0:1789760741.217776:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055b7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.217791:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760741.217796:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.217802:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.217816:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760741.244085:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973154 is committed 00002000:00010000:3.0:1789760741.244186:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23900160 -> 23904256 00010000:00010000:3.0:1789760741.244200:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.244213:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055b7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.244222:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.244229:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760741.244234:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.244243:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055b7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.244263:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055b7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.244272:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055b7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.244359:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493729152:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26638us (26848us total) trans 4294973154 rc 0/0 00000100:00100000:3.0:1789760741.244369:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5865 00000100:00100000:3.0:1789760741.244378:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751515660873 time 751558737925 state 0x2 ref 0 00000100:00100000:0.0:1789760741.247762:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760741.248020:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493729280 00000100:00100000:1.0:1789760741.248030:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493729280 00000100:00100000:1.0:1789760741.248089:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751515660873 time 751558737925 state 0x0 ref 1 00000100:00100000:1.0:1789760741.248108:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751558998341 time 751558737925 state 0x1 ref 1 00000100:00100000:1.0:1789760741.248138:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751558998341 time 751589173805 state 0x0 ref 1 00000100:00100000:1.0:1789760741.248151:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493729280:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760741.248202:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23904256, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:1.0:1789760741.248211:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e055be lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.248227:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760741.248231:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760741.248237:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e055be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.248262:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e055be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760741.283414:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973155 is committed 00002000:00010000:3.0:1789760741.283507:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23904256 -> 23908352 00010000:00010000:3.0:1789760741.283692:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e055be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.283706:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e055be lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.283717:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e055be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.283724:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760741.283730:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e055be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.283740:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e055be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.283760:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e055be lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.283769:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e055be lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.284102:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493729280:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35948us (36344us total) trans 4294973155 rc 0/0 00000100:00100000:3.0:1789760741.284113:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5866 00000100:00100000:3.0:1789760741.284122:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751558998341 time 751589173805 state 0x2 ref 0 00000100:00100000:2.0:1789760741.291777:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760741.294323:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493729408 00000100:00100000:0.0:1789760741.294338:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493729408 00000100:00100000:0.0:1789760741.294398:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751558998341 time 751589173805 state 0x0 ref 1 00000100:00100000:0.0:1789760741.294414:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751589434221 time 751589173805 state 0x1 ref 1 00000100:00100000:0.0:1789760741.294449:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751589434221 time 751635484496 state 0x0 ref 1 00000100:00100000:0.0:1789760741.294461:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493729408:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760741.294522:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23908352, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:0.0:1789760741.294531:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e055c5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.294547:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:0.0:1789760741.294551:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.294557:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e055c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.294573:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e055c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.337214:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973156 is committed 00002000:00010000:0.0:1789760741.337359:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23908352 -> 23912448 00010000:00010000:0.0:1789760741.337374:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e055c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.337388:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e055c5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.337398:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e055c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.337405:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760741.337409:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e055c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.337418:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e055c5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.337437:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e055c5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.337445:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e055c5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.337552:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493729408:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43084us (45776us total) trans 4294973156 rc 0/0 00000100:00100000:0.0:1789760741.337562:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5867 00000100:00100000:0.0:1789760741.337572:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751589434221 time 751635484496 state 0x2 ref 0 00000100:00100000:0.0:1789760741.343111:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760741.343331:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493729536 00000100:00100000:3.0:1789760741.343341:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493729536 00000100:00100000:3.0:1789760741.343399:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751589434221 time 751635484496 state 0x0 ref 1 00000100:00100000:3.0:1789760741.343414:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751635744912 time 751635484496 state 0x1 ref 1 00000100:00100000:3.0:1789760741.343449:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751635744912 time 751684484840 state 0x0 ref 1 00000100:00100000:3.0:1789760741.343461:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493729536:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760741.343519:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23912448, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:3.0:1789760741.343528:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055cc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.343553:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760741.343558:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.343565:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.343579:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760741.386522:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973157 is committed 00002000:00010000:3.0:1789760741.386526:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23912448 -> 23916544 00010000:00010000:3.0:1789760741.386544:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.386558:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055cc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.386566:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.386573:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760741.386578:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.386587:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055cc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.386949:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055cc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.387231:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055cc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.387324:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493729536:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43860us (44216us total) trans 4294973157 rc 0/0 00000100:00100000:3.0:1789760741.388820:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5868 00000100:00100000:3.0:1789760741.388831:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751635744912 time 751684484840 state 0x2 ref 0 00000100:00100000:1.0:1789760741.396051:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760741.396153:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493729664 00000100:00100000:3.0:1789760741.396165:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493729664 00000100:00100000:0.0:1789760741.396294:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751635744912 time 751684484840 state 0x0 ref 1 00000100:00100000:0.0:1789760741.396309:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751684745256 time 751684484840 state 0x1 ref 1 00000100:00100000:0.0:1789760741.396344:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751684745256 time 751737379533 state 0x0 ref 1 00000100:00100000:0.0:1789760741.396356:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493729664:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760741.396404:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23916544, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:0.0:1789760741.396412:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e055d3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.396427:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:0.0:1789760741.396431:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.396437:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e055d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.396456:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e055d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.437818:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973158 is committed 00002000:00010000:3.0:1789760741.437896:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23916544 -> 23920640 00010000:00010000:3.0:1789760741.437912:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e055d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.437925:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e055d3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.437944:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e055d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.437952:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760741.437957:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e055d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.438008:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e055d3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.438029:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e055d3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.438038:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e055d3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.438127:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493729664:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41768us (42080us total) trans 4294973158 rc 0/0 00000100:00100000:3.0:1789760741.438137:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5869 00000100:00100000:3.0:1789760741.438147:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751684745256 time 751737379533 state 0x2 ref 0 00000100:00100000:2.0:1789760741.444089:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760741.444249:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493729792 00000100:00100000:1.0:1789760741.444259:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493729792 00000100:00100000:1.0:1789760741.444314:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751684745256 time 751737379533 state 0x0 ref 1 00000100:00100000:1.0:1789760741.444329:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751737639949 time 751737379533 state 0x1 ref 1 00000100:00100000:1.0:1789760741.444358:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751737639949 time 751785394046 state 0x0 ref 1 00000100:00100000:1.0:1789760741.444371:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493729792:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760741.444418:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23920640, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:1.0:1789760741.444426:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e055da lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.444442:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760741.444447:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760741.444453:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e055da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.444466:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e055da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760741.488457:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23920640 -> 23924736 00010000:00010000:3.0:1789760741.488492:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e055da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.488510:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e055da lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.488521:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e055da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.488529:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00000001:00080000:2.0:1789760741.488613:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973159 is committed 00010000:00010000:3.0:1789760741.488669:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e055da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.488681:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e055da lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.488708:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e055da lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.488716:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e055da lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.489049:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493729792:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44670us (44960us total) trans 4294973159 rc 0/0 00000100:00100000:3.0:1789760741.489061:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5870 00000100:00100000:3.0:1789760741.489074:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751737639949 time 751785394046 state 0x2 ref 0 00000100:00100000:0.0:1789760741.495280:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760741.495373:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493729920 00000100:00100000:2.0:1789760741.495390:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493729920 00000100:00100000:2.0:1789760741.495467:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751737639949 time 751785394046 state 0x0 ref 1 00000100:00100000:2.0:1789760741.495497:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751785654462 time 751785394046 state 0x1 ref 1 00000100:00100000:2.0:1789760741.495531:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751785654462 time 751836566723 state 0x0 ref 1 00000100:00100000:2.0:1789760741.495546:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493729920:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760741.495611:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23924736, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:2.0:1789760741.495622:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e055e1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.495653:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760741.495658:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.495664:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e055e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.495681:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e055e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760741.532578:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973160 is committed 00002000:00010000:3.0:1789760741.532993:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23924736 -> 23928832 00010000:00010000:3.0:1789760741.533019:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e055e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.533035:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e055e1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.533050:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e055e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.533066:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760741.533072:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e055e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.533083:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e055e1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.533112:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e055e1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.533121:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e055e1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.533263:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493729920:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37712us (37984us total) trans 4294973160 rc 0/0 00000100:00100000:3.0:1789760741.533275:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5871 00000100:00100000:3.0:1789760741.533289:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751785654462 time 751836566723 state 0x2 ref 0 00000100:00100000:1.0:1789760741.538629:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760741.538863:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493730048 00000100:00100000:3.0:1789760741.538877:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493730048 00000100:00100000:3.0:1789760741.539067:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751785654462 time 751836566723 state 0x0 ref 1 00000100:00100000:3.0:1789760741.539089:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751836827139 time 751836566723 state 0x1 ref 1 00000100:00100000:3.0:1789760741.539126:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751836827139 time 751880161189 state 0x0 ref 1 00000100:00100000:3.0:1789760741.539146:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493730048:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760741.539218:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23928832, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:3.0:1789760741.539229:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055e8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.539250:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760741.539255:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.539262:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.539278:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760741.585088:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973161 is committed 00002000:00010000:3.0:1789760741.585230:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23928832 -> 23932928 00010000:00010000:3.0:1789760741.585250:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.585267:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055e8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.585277:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.585285:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760741.585291:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.585302:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055e8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.585331:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055e8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.585339:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055e8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.585469:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493730048:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46320us (46846us total) trans 4294973161 rc 0/0 00000100:00100000:3.0:1789760741.585726:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5872 00000100:00100000:3.0:1789760741.585739:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751836827139 time 751880161189 state 0x2 ref 0 00000100:00100000:2.0:1789760741.592824:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760741.594279:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493730176 00000100:00100000:3.0:1789760741.594304:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493730176 00000100:00100000:3.0:1789760741.595075:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751836827139 time 751880161189 state 0x0 ref 1 00000100:00100000:3.0:1789760741.595098:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751880421605 time 751880161189 state 0x1 ref 1 00000100:00100000:3.0:1789760741.595157:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751880421605 time 751936191766 state 0x0 ref 1 00000100:00100000:3.0:1789760741.595179:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493730176:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760741.595573:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23932928, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xb807 00010000:00010000:3.0:1789760741.595585:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055ef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.595614:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760741.595619:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.595627:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.595643:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760741.633261:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23932928 -> 23937024 00002000:00010000:3.0:1789760741.633273:0:13038:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760740 -> 1789760741 00002000:00010000:3.0:1789760741.633276:0:13038:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760740 -> 1789760741 00010000:00010000:3.0:1789760741.633294:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.633312:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055ef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.633322:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.633330:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760741.633335:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.633350:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.633379:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055ef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.633388:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055ef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.633607:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493730176:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38356us (40718us total) trans 4294973162 rc 0/0 00000100:00100000:3.0:1789760741.633624:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5873 00000100:00100000:3.0:1789760741.633637:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751880421605 time 751936191766 state 0x2 ref 0 00000001:00080000:0.0:1789760741.633737:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973162 is committed 00000100:00100000:0.0:1789760741.638618:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760741.638719:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493730304 00000100:00100000:1.0:1789760741.638732:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493730304 00000100:00100000:1.0:1789760741.638801:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751880421605 time 751936191766 state 0x0 ref 1 00000100:00100000:1.0:1789760741.638819:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751936452182 time 751936191766 state 0x1 ref 1 00000100:00100000:1.0:1789760741.638881:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751936452182 time 751979916593 state 0x0 ref 1 00000100:00100000:1.0:1789760741.638898:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493730304:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760741.639004:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23937024, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:1.0:1789760741.639015:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e055f6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.639038:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760741.639044:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760741.639052:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e055f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.639070:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e055f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760741.674670:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973163 is committed 00002000:00010000:3.0:1789760741.674806:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23937024 -> 23941120 00010000:00010000:3.0:1789760741.674826:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e055f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.674842:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e055f6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.674852:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e055f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.674860:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760741.674865:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e055f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.674883:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e055f6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.674912:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e055f6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.674921:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e055f6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.675647:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493730304:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36744us (37034us total) trans 4294973163 rc 0/0 00000100:00100000:3.0:1789760741.675662:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5874 00000100:00100000:3.0:1789760741.675676:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751936452182 time 751979916593 state 0x2 ref 0 00000100:00100000:2.0:1789760741.683830:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760741.683943:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493730432 00000100:00100000:3.0:1789760741.683956:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493730432 00000100:00100000:3.0:1789760741.684303:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751936452182 time 751979916593 state 0x0 ref 1 00000100:00100000:3.0:1789760741.684326:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751980177009 time 751979916593 state 0x1 ref 1 00000100:00100000:3.0:1789760741.684394:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751980177009 time 752025428843 state 0x0 ref 1 00000100:00100000:3.0:1789760741.684408:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493730432:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760741.684491:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23941120, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:3.0:1789760741.684502:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055fd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.684524:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760741.684529:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.684537:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.684554:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.737263:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973164 is committed 00002000:00010000:2.0:1789760741.744279:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23941120 -> 23945216 00010000:00010000:2.0:1789760741.744309:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.744325:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055fd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.744335:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.744343:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760741.744349:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.744366:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055fd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.744393:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055fd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.744926:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e055fd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.745385:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493730432:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60971us (61561us total) trans 4294973164 rc 0/0 00000100:00100000:2.0:1789760741.745408:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5875 00000100:00100000:2.0:1789760741.745420:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751980177009 time 752025428843 state 0x2 ref 0 00000100:00100000:2.0:1789760741.754536:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760741.758092:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493730560 00000100:00100000:3.0:1789760741.758114:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493730560 00000100:00100000:0.0:1789760741.758491:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 751980177009 time 752025428843 state 0x0 ref 1 00000100:00100000:0.0:1789760741.758512:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752025689259 time 752025428843 state 0x1 ref 1 00000100:00100000:0.0:1789760741.758591:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752025689259 time 752099625187 state 0x0 ref 1 00000100:00100000:0.0:1789760741.758609:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493730560:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760741.758677:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23945216, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:0.0:1789760741.758688:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05604 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.758709:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760741.758714:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.758722:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05604 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.758740:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05604 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760741.838835:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23945216 -> 23949312 00010000:00010000:3.0:1789760741.838857:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05604 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.838870:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05604 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.838878:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05604 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.838886:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760741.838891:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05604 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.838900:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05604 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.838925:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05604 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.838933:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05604 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.839065:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493730560:12345-192.168.203.60@tcp:4:dd.0 Request processed in 80456us (84540us total) trans 4294973165 rc 0/0 00000100:00100000:3.0:1789760741.839075:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5876 00000100:00100000:3.0:1789760741.839084:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752025689259 time 752099625187 state 0x2 ref 0 00000001:00080000:1.0:1789760741.839857:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973165 is committed 00000100:00100000:0.0:1789760741.842685:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760741.842779:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493730688 00000100:00100000:3.0:1789760741.842790:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493730688 00000100:00100000:3.0:1789760741.842841:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752025689259 time 752099625187 state 0x0 ref 1 00000100:00100000:3.0:1789760741.842855:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752099885603 time 752099625187 state 0x1 ref 1 00000100:00100000:3.0:1789760741.842932:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752099885603 time 752183966897 state 0x0 ref 1 00000100:00100000:3.0:1789760741.842945:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493730688:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760741.843086:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23949312, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:3.0:1789760741.843095:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0560b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.843111:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760741.843116:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.843122:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0560b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.843135:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0560b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760741.884799:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973166 is committed 00002000:00010000:3.0:1789760741.886769:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23949312 -> 23953408 00010000:00010000:3.0:1789760741.886798:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0560b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.886811:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0560b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.886819:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0560b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.886827:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760741.886832:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0560b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.886841:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0560b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.886861:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0560b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.886948:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0560b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.887162:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493730688:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44213us (44480us total) trans 4294973166 rc 0/0 00000100:00100000:3.0:1789760741.887174:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5877 00000100:00100000:3.0:1789760741.887184:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752099885603 time 752183966897 state 0x2 ref 0 00000100:00100000:3.0:1789760741.892402:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760741.892532:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493730816 00000100:00100000:1.0:1789760741.892551:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493730816 00000100:00100000:1.0:1789760741.892657:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752099885603 time 752183966897 state 0x0 ref 1 00000100:00100000:1.0:1789760741.892680:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752184227313 time 752183966897 state 0x1 ref 1 00000100:00100000:1.0:1789760741.892729:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752184227313 time 752233762738 state 0x0 ref 1 00000100:00100000:1.0:1789760741.892750:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8556a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493730816:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760741.892829:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23953408, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:1.0:1789760741.892841:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05612 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.892863:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760741.892869:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760741.892877:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05612 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.892898:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05612 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760741.943540:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973167 is committed 00002000:00010000:1.0:1789760741.943721:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23953408 -> 23957504 00010000:00010000:1.0:1789760741.943736:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05612 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.943749:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05612 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.943758:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05612 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.943766:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760741.943770:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05612 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.943779:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05612 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.943799:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05612 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.943807:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05612 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760741.946750:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8556a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493730816:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53990us (54342us total) trans 4294973167 rc 0/0 00000100:00100000:1.0:1789760741.946765:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5878 00000100:00100000:1.0:1789760741.946775:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752184227313 time 752233762738 state 0x2 ref 0 00010000:00010000:1.0:1789760741.950654:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789760741.960414:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760741.960694:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493730944 00000100:00100000:0.0:1789760741.960704:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493730944 00000100:00100000:0.0:1789760741.960755:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752184227313 time 752233762738 state 0x0 ref 1 00000100:00100000:0.0:1789760741.960768:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752234023154 time 752233762738 state 0x1 ref 1 00000100:00100000:0.0:1789760741.960797:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752234023154 time 752301832307 state 0x0 ref 1 00000100:00100000:0.0:1789760741.960810:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493730944:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760741.960872:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23957504, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:0.0:1789760741.960881:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05619 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.960907:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760741.960912:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.960918:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05619 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.960930:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05619 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760741.999428:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973168 is committed 00002000:00010000:0.0:1789760741.999612:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23957504 -> 23961600 00010000:00010000:0.0:1789760741.999626:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05619 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.999644:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05619 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.999652:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05619 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.999658:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760741.999663:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05619 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.999671:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05619 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.999691:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05619 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.999699:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05619 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.999852:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493730944:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38973us (39377us total) trans 4294973168 rc 0/0 00000100:00100000:0.0:1789760741.999864:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5879 00000100:00100000:0.0:1789760741.999875:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752234023154 time 752301832307 state 0x2 ref 0 00000100:00100000:2.0:1789760742.004044:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760742.004297:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493731072 00000100:00100000:3.0:1789760742.004308:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493731072 00000100:00100000:3.0:1789760742.004360:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752234023154 time 752301832307 state 0x0 ref 1 00000100:00100000:3.0:1789760742.004375:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752302092723 time 752301832307 state 0x1 ref 1 00000100:00100000:3.0:1789760742.004406:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752302092723 time 752345441719 state 0x0 ref 1 00000100:00100000:3.0:1789760742.004418:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493731072:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760742.004466:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23961600, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:3.0:1789760742.004489:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05620 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.004505:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760742.004509:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.004516:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05620 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.004530:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05620 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760742.037925:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973169 is committed 00002000:00010000:2.0:1789760742.038211:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23961600 -> 23965696 00010000:00010000:2.0:1789760742.038226:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05620 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.038238:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05620 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.038253:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05620 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.038261:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760742.038267:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05620 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.038276:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05620 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.038296:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05620 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.038305:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05620 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.038425:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493731072:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34004us (34386us total) trans 4294973169 rc 0/0 00000100:00100000:2.0:1789760742.038436:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5880 00000100:00100000:2.0:1789760742.038446:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752302092723 time 752345441719 state 0x2 ref 0 00000100:00100000:2.0:1789760742.046681:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760742.046757:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493731200 00000100:00100000:3.0:1789760742.046767:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493731200 00000100:00100000:3.0:1789760742.046819:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752302092723 time 752345441719 state 0x0 ref 1 00000100:00100000:3.0:1789760742.046834:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752345702135 time 752345441719 state 0x1 ref 1 00000100:00100000:3.0:1789760742.046904:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752345702135 time 752387939397 state 0x0 ref 1 00000100:00100000:3.0:1789760742.046918:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493731200:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760742.047008:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23965696, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:3.0:1789760742.047016:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05627 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.047032:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760742.047037:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.047043:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05627 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.047056:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05627 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760742.084684:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23965696 -> 23969792 00010000:00010000:3.0:1789760742.084705:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05627 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.084719:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05627 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.084727:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05627 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.084735:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760742.084741:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05627 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.084749:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05627 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.084769:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05627 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.084777:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05627 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.084870:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493731200:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37950us (38193us total) trans 4294973170 rc 0/0 00000100:00100000:3.0:1789760742.084885:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5881 00000100:00100000:3.0:1789760742.084894:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752345702135 time 752387939397 state 0x2 ref 0 00000001:00080000:3.0:1789760742.085050:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973170 is committed 00000100:00100000:2.0:1789760742.088123:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760742.088328:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493731328 00000100:00100000:1.0:1789760742.088340:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493731328 00000100:00100000:1.0:1789760742.088402:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752345702135 time 752387939397 state 0x0 ref 1 00000100:00100000:1.0:1789760742.088417:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752388199813 time 752387939397 state 0x1 ref 1 00000100:00100000:1.0:1789760742.088490:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752388199813 time 752429524540 state 0x0 ref 1 00000100:00100000:1.0:1789760742.088508:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493731328:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760742.088614:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23969792, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:1.0:1789760742.088621:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0562e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.088635:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760742.088638:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.088642:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0562e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.088652:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0562e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.125301:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973171 is committed 00002000:00010000:3.0:1789760742.125540:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23969792 -> 23973888 00010000:00010000:3.0:1789760742.125555:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0562e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.125567:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0562e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.125577:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0562e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.125584:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760742.125589:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0562e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.125597:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0562e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.125614:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0562e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.125622:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0562e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.125719:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493731328:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37209us (37604us total) trans 4294973171 rc 0/0 00000100:00100000:3.0:1789760742.125729:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5882 00000100:00100000:3.0:1789760742.125739:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752388199813 time 752429524540 state 0x2 ref 0 00000100:00100000:0.0:1789760742.129136:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760742.129578:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493731456 00000100:00100000:1.0:1789760742.129588:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493731456 00000100:00100000:1.0:1789760742.129650:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752388199813 time 752429524540 state 0x0 ref 1 00000100:00100000:1.0:1789760742.129664:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752429784956 time 752429524540 state 0x1 ref 1 00000100:00100000:1.0:1789760742.129714:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752429784956 time 752470750716 state 0x0 ref 1 00000100:00100000:1.0:1789760742.129731:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493731456:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760742.129777:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23973888, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:1.0:1789760742.129784:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05635 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.129796:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760742.129799:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.129804:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05635 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.129815:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05635 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760742.162565:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23973888 -> 23977984 00010000:00010000:2.0:1789760742.162583:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05635 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.162597:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05635 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.162606:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05635 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.162615:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760742.162619:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05635 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.162628:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05635 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.162645:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05635 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.162653:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05635 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.162753:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493731456:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33025us (33621us total) trans 4294973172 rc 0/0 00000100:00100000:2.0:1789760742.162764:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5883 00000100:00100000:2.0:1789760742.162773:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752429784956 time 752470750716 state 0x2 ref 0 00000001:00080000:3.0:1789760742.163295:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973172 is committed 00000100:00100000:2.0:1789760742.172208:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760742.172530:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493731584 00000100:00100000:3.0:1789760742.172698:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493731584 00000100:00100000:3.0:1789760742.172752:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752429784956 time 752470750716 state 0x0 ref 1 00000100:00100000:3.0:1789760742.172767:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752471011132 time 752470750716 state 0x1 ref 1 00000100:00100000:3.0:1789760742.172802:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752471011132 time 752513837492 state 0x0 ref 1 00000100:00100000:3.0:1789760742.172813:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493731584:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760742.172861:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23977984, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:3.0:1789760742.172868:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0563c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.172882:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760742.172886:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.172892:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0563c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.172905:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0563c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.217370:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973173 is committed 00002000:00010000:3.0:1789760742.217462:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23977984 -> 23982080 00010000:00010000:3.0:1789760742.217491:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0563c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.217505:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0563c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.217515:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0563c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.217523:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760742.217528:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0563c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.217538:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0563c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.217559:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0563c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.217567:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0563c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.217747:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493731584:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44930us (45541us total) trans 4294973173 rc 0/0 00000100:00100000:3.0:1789760742.217762:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5884 00000100:00100000:3.0:1789760742.217772:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752471011132 time 752513837492 state 0x2 ref 0 00000100:00100000:0.0:1789760742.222240:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760742.222289:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493731712 00000100:00100000:3.0:1789760742.222299:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493731712 00000100:00100000:3.0:1789760742.222356:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752471011132 time 752513837492 state 0x0 ref 1 00000100:00100000:3.0:1789760742.222371:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752514097908 time 752513837492 state 0x1 ref 1 00000100:00100000:3.0:1789760742.222401:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752514097908 time 752563437000 state 0x0 ref 1 00000100:00100000:3.0:1789760742.222412:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493731712:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760742.222458:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23982080, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:3.0:1789760742.222466:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05643 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.222643:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760742.222648:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.222654:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05643 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.222668:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05643 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760742.249291:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973174 is committed 00002000:00010000:3.0:1789760742.249496:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23982080 -> 23986176 00010000:00010000:3.0:1789760742.249511:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05643 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.249646:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05643 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.249659:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05643 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.249667:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760742.249672:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05643 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.249682:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05643 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.249749:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05643 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.249758:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05643 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.249854:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493731712:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27437us (27618us total) trans 4294973174 rc 0/0 00000100:00100000:3.0:1789760742.249865:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5885 00000100:00100000:3.0:1789760742.249875:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752514097908 time 752563437000 state 0x2 ref 0 00000100:00100000:1.0:1789760742.259096:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760742.259152:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493731840 00000100:00100000:3.0:1789760742.259164:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493731840 00000100:00100000:3.0:1789760742.259309:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752514097908 time 752563437000 state 0x0 ref 1 00000100:00100000:3.0:1789760742.259325:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752563697416 time 752563437000 state 0x1 ref 1 00000100:00100000:2.0:1789760742.259422:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752563697416 time 752600454738 state 0x0 ref 1 00000100:00100000:2.0:1789760742.259439:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493731840:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760742.259580:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23986176, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:2.0:1789760742.259589:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0564a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.259604:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760742.259608:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.259614:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0564a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.259637:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0564a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760742.305943:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23986176 -> 23990272 00000001:00080000:0.0:1789760742.306095:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973175 is committed 00010000:00010000:2.0:1789760742.306175:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0564a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.306189:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0564a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.306199:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0564a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.306207:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760742.306212:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0564a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.306222:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0564a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.306242:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0564a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.306250:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0564a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.306349:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493731840:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46909us (47258us total) trans 4294973175 rc 0/0 00000100:00100000:2.0:1789760742.306359:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5886 00000100:00100000:2.0:1789760742.306369:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752563697416 time 752600454738 state 0x2 ref 0 00000100:00100000:2.0:1789760742.317325:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760742.317576:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493731968 00000100:00100000:1.0:1789760742.317587:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493731968 00000100:00100000:1.0:1789760742.317644:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752563697416 time 752600454738 state 0x0 ref 1 00000100:00100000:1.0:1789760742.317660:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752600715154 time 752600454738 state 0x1 ref 1 00000100:00100000:1.0:1789760742.317741:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752600715154 time 752658776075 state 0x0 ref 1 00000100:00100000:1.0:1789760742.317754:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493731968:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760742.317802:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23990272, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:1.0:1789760742.317813:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05651 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.317830:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760742.317835:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760742.317842:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05651 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.317858:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05651 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.361937:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973176 is committed 00002000:00010000:3.0:1789760742.362232:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23990272 -> 23994368 00010000:00010000:3.0:1789760742.362247:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05651 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.362261:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05651 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.362270:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05651 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.362278:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760742.362284:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05651 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.362294:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05651 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.362313:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05651 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.362321:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05651 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.362424:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493731968:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44667us (45105us total) trans 4294973176 rc 0/0 00000100:00100000:3.0:1789760742.362435:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5887 00000100:00100000:3.0:1789760742.362444:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752600715154 time 752658776075 state 0x2 ref 0 00000100:00100000:3.0:1789760742.367726:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760742.367874:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493732096 00000100:00100000:1.0:1789760742.367883:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493732096 00000100:00100000:1.0:1789760742.367938:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752600715154 time 752658776075 state 0x0 ref 1 00000100:00100000:1.0:1789760742.367956:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752659036491 time 752658776075 state 0x1 ref 1 00000100:00100000:0.0:1789760742.368244:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752659036491 time 752709276693 state 0x0 ref 1 00000100:00100000:0.0:1789760742.368260:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493732096:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760742.368312:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23994368, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:0.0:1789760742.368322:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05658 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.368339:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760742.368344:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.368350:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05658 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.368365:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05658 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.405399:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973177 is committed 00002000:00010000:3.0:1789760742.405521:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23994368 -> 23998464 00010000:00010000:3.0:1789760742.405641:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05658 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.405654:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05658 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.405662:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05658 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.405669:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760742.405674:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05658 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.405683:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05658 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.405703:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05658 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.405716:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05658 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.405850:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493732096:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37588us (38130us total) trans 4294973177 rc 0/0 00000100:00100000:3.0:1789760742.405862:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5888 00000100:00100000:3.0:1789760742.405872:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752659036491 time 752709276693 state 0x2 ref 0 00000100:00100000:1.0:1789760742.410308:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760742.410389:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493732224 00000100:00100000:3.0:1789760742.410401:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493732224 00000100:00100000:3.0:1789760742.410456:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752659036491 time 752709276693 state 0x0 ref 1 00000100:00100000:3.0:1789760742.410485:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752709537109 time 752709276693 state 0x1 ref 1 00000100:00100000:3.0:1789760742.410517:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752709537109 time 752751552224 state 0x0 ref 1 00000100:00100000:3.0:1789760742.410530:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493732224:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760742.410579:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23998464, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:3.0:1789760742.410588:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0565f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.410604:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760742.410608:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.410614:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0565f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.410633:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0565f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760742.436030:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 23998464 -> 24002560 00010000:00010000:2.0:1789760742.436051:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0565f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.436064:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0565f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.436075:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0565f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.436089:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760742.436094:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0565f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.436103:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0565f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.436123:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0565f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.436131:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0565f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.436218:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493732224:12345-192.168.203.60@tcp:4:dd.0 Request processed in 25685us (25915us total) trans 4294973178 rc 0/0 00000100:00100000:2.0:1789760742.436229:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5889 00000100:00100000:2.0:1789760742.436239:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752709537109 time 752751552224 state 0x2 ref 0 00000001:00080000:3.0:1789760742.436761:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973178 is committed 00000100:00100000:2.0:1789760742.441233:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760742.441314:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493732352 00000100:00100000:3.0:1789760742.441324:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493732352 00000100:00100000:3.0:1789760742.441377:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752709537109 time 752751552224 state 0x0 ref 1 00000100:00100000:3.0:1789760742.441392:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752751812640 time 752751552224 state 0x1 ref 1 00000100:00100000:3.0:1789760742.441414:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752751812640 time 752782449686 state 0x0 ref 1 00000100:00100000:3.0:1789760742.441427:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493732352:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760742.441484:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24002560, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:3.0:1789760742.441493:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05666 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.441509:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760742.441514:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.441520:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05666 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.441544:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05666 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.469689:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973179 is committed 00002000:00010000:3.0:1789760742.469794:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24002560 -> 24006656 00010000:00010000:3.0:1789760742.469812:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05666 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.469824:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05666 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.469833:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05666 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.469839:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760742.469844:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05666 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.469852:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05666 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.469870:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05666 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.469878:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05666 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.470000:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493732352:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28570us (28771us total) trans 4294973179 rc 0/0 00000100:00100000:3.0:1789760742.470012:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5890 00000100:00100000:3.0:1789760742.470021:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752751812640 time 752782449686 state 0x2 ref 0 00000100:00100000:1.0:1789760742.473050:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760742.473147:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493732480 00000100:00100000:3.0:1789760742.473157:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493732480 00000100:00100000:3.0:1789760742.473205:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752751812640 time 752782449686 state 0x0 ref 1 00000100:00100000:3.0:1789760742.473221:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752782710102 time 752782449686 state 0x1 ref 1 00000100:00100000:3.0:1789760742.473242:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752782710102 time 752814278272 state 0x0 ref 1 00000100:00100000:3.0:1789760742.473254:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493732480:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760742.473299:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24006656, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:3.0:1789760742.473307:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0566d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.473321:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760742.473325:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.473330:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0566d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.473343:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0566d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.503408:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760742.503494:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493732736 00000100:00100000:3.0:1789760742.503504:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493732736 00000100:00100000:3.0:1789760742.503575:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 482 00000100:00100000:3.0:1789760742.503597:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2369:x1876699493732736:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:3.0:1789760742.503681:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2369:x1876699493732736:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 97us (282us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760742.503688:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 482 00000100:00080000:2.0:1789760742.503938:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760742.504460:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760742.504520:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493732608 00000100:00100000:0.0:1789760742.504525:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493732608 00000100:00100000:0.0:1789760742.504564:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 399 00000100:00100000:0.0:1789760742.504572:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d4d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2369:x1876699493732608:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:0.0:1789760742.504621:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d4d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2369:x1876699493732608:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 49us (162us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760742.504631:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 399 00000100:00100000:3.0:1789760742.504922:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760742.505169:0:10519:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493732864 00000100:00100000:0.0:1789760742.505175:0:10519:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493732864 00000100:00080000:2.0:1789760742.505209:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760742.505669:0:10519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 483 00000100:00100000:0.0:1789760742.505683:0:10519:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe84700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2369:x1876699493732864:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:0.0:1789760742.505742:0:10519:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe84700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2369:x1876699493732864:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 64us (821us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760742.505750:0:10519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 483 00000001:00080000:1.0:1789760742.512227:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973180 is committed 00002000:00010000:2.0:1789760742.512415:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24006656 -> 24010752 00010000:00010000:2.0:1789760742.512431:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0566d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.512451:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0566d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.512460:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0566d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.512467:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760742.512482:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0566d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.512492:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0566d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.512510:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0566d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.512518:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0566d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.512675:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493732480:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39417us (39627us total) trans 4294973180 rc 0/0 00000100:00100000:2.0:1789760742.512684:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5891 00000100:00100000:2.0:1789760742.512693:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752782710102 time 752814278272 state 0x2 ref 0 00000100:00100000:3.0:1789760742.519881:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760742.519994:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493732992 00000100:00100000:0.0:1789760742.520004:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493732992 00000100:00100000:0.0:1789760742.520058:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752782710102 time 752814278272 state 0x0 ref 1 00000100:00100000:0.0:1789760742.520073:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752814538688 time 752814278272 state 0x1 ref 1 00000100:00100000:0.0:1789760742.520105:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752814538688 time 752861140409 state 0x0 ref 1 00000100:00100000:0.0:1789760742.520118:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493732992:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760742.520166:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24010752, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:0.0:1789760742.520175:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05674 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.520190:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760742.520195:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.520202:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05674 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.520222:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05674 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760742.544758:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973181 is committed 00002000:00010000:1.0:1789760742.544837:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24010752 -> 24014848 00010000:00010000:1.0:1789760742.544852:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05674 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.544863:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05674 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.544871:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05674 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.544877:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760742.544882:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05674 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.544890:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05674 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.544909:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05674 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.544916:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05674 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.545112:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493732992:12345-192.168.203.60@tcp:4:dd.0 Request processed in 24991us (25236us total) trans 4294973181 rc 0/0 00000100:00100000:1.0:1789760742.545126:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5892 00000100:00100000:1.0:1789760742.545135:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752814538688 time 752861140409 state 0x2 ref 0 00000100:00100000:3.0:1789760742.553779:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760742.553887:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493733120 00000100:00100000:2.0:1789760742.553898:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493733120 00000100:00100000:2.0:1789760742.553997:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752814538688 time 752861140409 state 0x0 ref 1 00000100:00100000:2.0:1789760742.554013:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752861400825 time 752861140409 state 0x1 ref 1 00000100:00100000:2.0:1789760742.554050:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752861400825 time 752895085671 state 0x0 ref 1 00000100:00100000:2.0:1789760742.554063:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493733120:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760742.554109:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24014848, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:2.0:1789760742.554118:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0567b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.554400:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760742.554406:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.554413:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0567b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.554428:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0567b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760742.598760:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973182 is committed 00002000:00010000:1.0:1789760742.599116:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24014848 -> 24018944 00010000:00010000:1.0:1789760742.599207:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0567b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.599221:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0567b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.599229:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0567b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.599237:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760742.599242:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0567b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.599251:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0567b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.599272:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0567b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.599280:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0567b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.599482:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493733120:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45406us (45738us total) trans 4294973182 rc 0/0 00000100:00100000:1.0:1789760742.599495:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5893 00000100:00100000:1.0:1789760742.599505:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752861400825 time 752895085671 state 0x2 ref 0 00000100:00100000:0.0:1789760742.606308:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760742.606402:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493733248 00000100:00100000:3.0:1789760742.606412:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493733248 00000100:00100000:3.0:1789760742.606465:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752861400825 time 752895085671 state 0x0 ref 1 00000100:00100000:3.0:1789760742.606490:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752895346087 time 752895085671 state 0x1 ref 1 00000100:00100000:3.0:1789760742.606520:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752895346087 time 752947555912 state 0x0 ref 1 00000100:00100000:3.0:1789760742.606533:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493733248:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760742.606582:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24018944, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xb807 00010000:00010000:3.0:1789760742.606590:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05682 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.606605:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760742.606610:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.606616:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05682 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.606634:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05682 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760742.660150:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973183 is committed 00002000:00010000:1.0:1789760742.660242:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24018944 -> 24023040 00002000:00010000:1.0:1789760742.660247:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760741 -> 1789760742 00002000:00010000:1.0:1789760742.660249:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760741 -> 1789760742 00010000:00010000:1.0:1789760742.660262:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05682 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.660274:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05682 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.660283:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05682 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.660289:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760742.660293:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05682 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.660301:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05682 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.660323:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05682 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.660330:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05682 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.660456:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493733248:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53921us (54153us total) trans 4294973183 rc 0/0 00000100:00100000:1.0:1789760742.660467:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5894 00000100:00100000:1.0:1789760742.660486:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752895346087 time 752947555912 state 0x2 ref 0 00000100:00100000:3.0:1789760742.664513:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760742.664764:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493733376 00000100:00100000:2.0:1789760742.664775:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493733376 00000100:00100000:2.0:1789760742.664837:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752895346087 time 752947555912 state 0x0 ref 1 00000100:00100000:2.0:1789760742.664852:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752947816328 time 752947555912 state 0x1 ref 1 00000100:00100000:2.0:1789760742.664916:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752947816328 time 753005950657 state 0x0 ref 1 00000100:00100000:2.0:1789760742.664932:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493733376:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760742.665013:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24023040, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:2.0:1789760742.665022:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05689 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.665038:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760742.665042:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.665048:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05689 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.665062:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05689 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.699554:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973184 is committed 00002000:00010000:0.0:1789760742.699851:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24023040 -> 24027136 00010000:00010000:0.0:1789760742.699866:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05689 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.699877:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05689 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.699885:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05689 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.699892:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760742.699897:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05689 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.699904:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05689 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.699923:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05689 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.699930:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05689 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760742.703344:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493733376:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35168us (35593us total) trans 4294973184 rc 0/0 00000100:00100000:0.0:1789760742.703366:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5895 00000100:00100000:0.0:1789760742.703378:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752947816328 time 753005950657 state 0x2 ref 0 00000100:00100000:1.0:1789760742.706641:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760742.706726:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493733504 00000100:00100000:0.0:1789760742.706738:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493733504 00000100:00100000:0.0:1789760742.706796:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 752947816328 time 753005950657 state 0x0 ref 1 00000100:00100000:0.0:1789760742.706812:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753006211073 time 753005950657 state 0x1 ref 1 00000100:00100000:0.0:1789760742.706844:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753006211073 time 753047879707 state 0x0 ref 1 00000100:00100000:0.0:1789760742.706858:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493733504:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760742.706912:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24027136, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:0.0:1789760742.706923:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05690 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.706942:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760742.706947:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.706997:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05690 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.707013:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05690 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760742.739318:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24027136 -> 24031232 00010000:00010000:0.0:1789760742.739338:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05690 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.739351:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05690 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.739361:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05690 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.739369:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760742.739374:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05690 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.739384:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05690 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.739404:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05690 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.739412:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05690 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760742.739530:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493733504:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32663us (32994us total) trans 4294973185 rc 0/0 00000100:00100000:0.0:1789760742.739544:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5896 00000100:00100000:0.0:1789760742.739554:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753006211073 time 753047879707 state 0x2 ref 0 00000001:00080000:2.0:1789760742.740938:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973185 is committed 00000100:00100000:1.0:1789760742.744274:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760742.744365:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493733632 00000100:00100000:2.0:1789760742.744375:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493733632 00000100:00100000:2.0:1789760742.744429:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753006211073 time 753047879707 state 0x0 ref 1 00000100:00100000:2.0:1789760742.744445:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753048140123 time 753047879707 state 0x1 ref 1 00000100:00100000:2.0:1789760742.744573:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753048140123 time 753085557477 state 0x0 ref 1 00000100:00100000:2.0:1789760742.744589:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb06850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493733632:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760742.744639:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24031232, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:2.0:1789760742.744648:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05697 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.744663:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760742.744667:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.744673:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05697 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.744686:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05697 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.769094:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973186 is committed 00002000:00010000:0.0:1789760742.769169:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24031232 -> 24035328 00010000:00010000:0.0:1789760742.769186:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05697 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.769203:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05697 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.769212:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05697 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.769220:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760742.769224:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05697 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.769234:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05697 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.769254:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05697 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.769263:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05697 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760742.769362:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb06850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493733632:12345-192.168.203.60@tcp:4:dd.0 Request processed in 24772us (25092us total) trans 4294973186 rc 0/0 00000100:00100000:0.0:1789760742.769372:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5897 00000100:00100000:0.0:1789760742.769381:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753048140123 time 753085557477 state 0x2 ref 0 00000100:00100000:1.0:1789760742.774321:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760742.774555:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493733760 00000100:00100000:3.0:1789760742.774565:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493733760 00000100:00100000:3.0:1789760742.774620:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753048140123 time 753085557477 state 0x0 ref 1 00000100:00100000:3.0:1789760742.774635:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753085817893 time 753085557477 state 0x1 ref 1 00000100:00100000:3.0:1789760742.774664:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753085817893 time 753115700252 state 0x0 ref 1 00000100:00100000:3.0:1789760742.774677:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493733760:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760742.774724:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24035328, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:3.0:1789760742.774733:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0569e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.774867:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:3.0:1789760742.774872:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.774878:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0569e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.774892:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0569e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.807765:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973187 is committed 00002000:00010000:0.0:1789760742.808463:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24035328 -> 24039424 00010000:00010000:0.0:1789760742.808491:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0569e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.808503:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0569e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.808511:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0569e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.808518:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760742.808523:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0569e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.808533:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0569e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.808555:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0569e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.808569:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0569e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760742.808664:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493733760:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33984us (34347us total) trans 4294973187 rc 0/0 00000100:00100000:0.0:1789760742.808675:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5898 00000100:00100000:0.0:1789760742.808685:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753085817893 time 753115700252 state 0x2 ref 0 00000100:00100000:1.0:1789760742.816620:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760742.817483:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493733888 00000100:00100000:0.0:1789760742.817493:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493733888 00000100:00100000:0.0:1789760742.817550:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753085817893 time 753115700252 state 0x0 ref 1 00000100:00100000:0.0:1789760742.817565:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753115960668 time 753115700252 state 0x1 ref 1 00000100:00100000:0.0:1789760742.817595:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753115960668 time 753158630813 state 0x0 ref 1 00000100:00100000:0.0:1789760742.817607:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493733888:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760742.817655:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24039424, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:0.0:1789760742.817665:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e056a5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.817679:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760742.817683:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.817689:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e056a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.817707:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e056a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.858692:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973188 is committed 00002000:00010000:0.0:1789760742.858902:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24039424 -> 24043520 00010000:00010000:0.0:1789760742.858916:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e056a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.858930:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e056a5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.859102:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e056a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.859338:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760742.859343:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e056a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.859354:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e056a5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.859375:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e056a5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.859382:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e056a5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760742.859642:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493733888:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42031us (43027us total) trans 4294973188 rc 0/0 00000100:00100000:0.0:1789760742.859653:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5899 00000100:00100000:0.0:1789760742.859663:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753115960668 time 753158630813 state 0x2 ref 0 00000100:00100000:1.0:1789760742.864555:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760742.864668:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493734016 00000100:00100000:0.0:1789760742.864678:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493734016 00000100:00100000:0.0:1789760742.864731:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753115960668 time 753158630813 state 0x0 ref 1 00000100:00100000:0.0:1789760742.864746:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753158891229 time 753158630813 state 0x1 ref 1 00000100:00100000:0.0:1789760742.864779:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753158891229 time 753205813685 state 0x0 ref 1 00000100:00100000:0.0:1789760742.864792:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493734016:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760742.864845:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24043520, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:0.0:1789760742.864854:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e056ac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.864869:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760742.864873:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.864879:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e056ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.864892:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e056ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760742.900183:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24043520 -> 24047616 00010000:00010000:0.0:1789760742.900203:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e056ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.900217:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e056ac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.900227:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e056ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.900234:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760742.900240:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e056ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.900250:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e056ac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.900271:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e056ac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.900278:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e056ac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760742.900376:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493734016:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35578us (35823us total) trans 4294973189 rc 0/0 00000100:00100000:0.0:1789760742.900390:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5900 00000100:00100000:0.0:1789760742.900400:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753158891229 time 753205813685 state 0x2 ref 0 00000001:00080000:2.0:1789760742.900484:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973189 is committed 00000100:00100000:1.0:1789760742.905275:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760742.905398:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493734144 00000100:00100000:2.0:1789760742.905408:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493734144 00000100:00100000:2.0:1789760742.905465:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753158891229 time 753205813685 state 0x0 ref 1 00000100:00100000:2.0:1789760742.905492:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753206074101 time 753205813685 state 0x1 ref 1 00000100:00100000:2.0:1789760742.905526:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753206074101 time 753246555729 state 0x0 ref 1 00000100:00100000:2.0:1789760742.905539:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493734144:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760742.905588:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24047616, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:2.0:1789760742.905597:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e056b3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.905617:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760742.905621:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.905628:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e056b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.905641:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e056b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760742.935808:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973190 is committed 00002000:00010000:2.0:1789760742.936208:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24047616 -> 24051712 00010000:00010000:2.0:1789760742.936222:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e056b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.936233:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e056b3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.936242:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e056b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.936250:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760742.936254:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e056b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.936263:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e056b3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.936283:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e056b3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.936292:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e056b3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.936373:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493734144:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30832us (31102us total) trans 4294973190 rc 0/0 00000100:00100000:2.0:1789760742.936383:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5901 00000100:00100000:2.0:1789760742.936392:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753206074101 time 753246555729 state 0x2 ref 0 00000100:00100000:1.0:1789760742.944836:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760742.945366:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493734272 00000100:00100000:3.0:1789760742.945377:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493734272 00000100:00100000:3.0:1789760742.945432:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753206074101 time 753246555729 state 0x0 ref 1 00000100:00100000:3.0:1789760742.945446:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753246816145 time 753246555729 state 0x1 ref 1 00000100:00100000:3.0:1789760742.945527:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753246816145 time 753286557362 state 0x0 ref 1 00000100:00100000:3.0:1789760742.945539:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493734272:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760742.945590:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24051712, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:3.0:1789760742.945599:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e056ba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.945616:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:3.0:1789760742.945621:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.945627:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e056ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.945640:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e056ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.970151:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760742.983284:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973191 is committed 00002000:00010000:3.0:1789760742.983443:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24051712 -> 24055808 00010000:00010000:3.0:1789760742.983457:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e056ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.983486:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e056ba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.983495:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e056ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.983502:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760742.983507:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e056ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.983516:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e056ba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.983544:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e056ba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.983551:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e056ba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.983646:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493734272:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38103us (38813us total) trans 4294973191 rc 0/0 00000100:00100000:3.0:1789760742.983657:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5902 00000100:00100000:3.0:1789760742.983667:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753246816145 time 753286557362 state 0x2 ref 0 00000100:00100000:1.0:1789760742.989566:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760742.989662:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493734400 00000100:00100000:0.0:1789760742.989672:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493734400 00000100:00100000:0.0:1789760742.989729:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753246816145 time 753286557362 state 0x0 ref 1 00000100:00100000:0.0:1789760742.989747:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753286817778 time 753286557362 state 0x1 ref 1 00000100:00100000:0.0:1789760742.989778:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753286817778 time 753330813627 state 0x0 ref 1 00000100:00100000:0.0:1789760742.989790:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493734400:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760742.989837:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24055808, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:0.0:1789760742.989846:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e056c1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.989863:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760742.989868:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.989874:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e056c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.989888:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e056c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760743.040415:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24055808 -> 24059904 00010000:00010000:1.0:1789760743.040435:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e056c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.040448:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e056c1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.040462:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e056c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.040479:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760743.040484:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e056c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.040494:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e056c1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.040515:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e056c1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.040523:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e056c1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760743.040617:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493734400:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50824us (51056us total) trans 4294973192 rc 0/0 00000100:00100000:1.0:1789760743.040627:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5903 00000100:00100000:1.0:1789760743.040637:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753286817778 time 753330813627 state 0x2 ref 0 00000001:00080000:3.0:1789760743.043765:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973192 is committed 00000100:00100000:0.0:1789760743.044674:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760743.044770:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493734528 00000100:00100000:1.0:1789760743.044780:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493734528 00000100:00100000:1.0:1789760743.044855:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753286817778 time 753330813627 state 0x0 ref 1 00000100:00100000:1.0:1789760743.044871:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753331074043 time 753330813627 state 0x1 ref 1 00000100:00100000:1.0:1789760743.044943:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753331074043 time 753385978538 state 0x0 ref 1 00000100:00100000:1.0:1789760743.044957:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493734528:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760743.045046:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24059904, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:1.0:1789760743.045054:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e056c8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.045069:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760743.045074:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760743.045081:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e056c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.045094:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e056c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760743.045533:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789760743.078936:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24059904 -> 24064000 00000001:00080000:3.0:1789760743.078941:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973193 is committed 00010000:00010000:1.0:1789760743.079031:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e056c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.079044:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e056c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.079054:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e056c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.079061:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760743.079065:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e056c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.079074:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e056c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.079093:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e056c8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.079100:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e056c8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760743.079195:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493734528:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34235us (34595us total) trans 4294973193 rc 0/0 00000100:00100000:1.0:1789760743.079205:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5904 00000100:00100000:1.0:1789760743.079215:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753331074043 time 753385978538 state 0x2 ref 0 00000100:00100000:0.0:1789760743.083151:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760743.083216:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493734656 00000100:00100000:2.0:1789760743.083234:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493734656 00000100:00100000:2.0:1789760743.083301:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753331074043 time 753385978538 state 0x0 ref 1 00000100:00100000:2.0:1789760743.083317:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753386238954 time 753385978538 state 0x1 ref 1 00000100:00100000:2.0:1789760743.083358:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753386238954 time 753424393158 state 0x0 ref 1 00000100:00100000:2.0:1789760743.083373:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493734656:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760743.083423:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24064000, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:2.0:1789760743.083432:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e056cf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.083451:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760743.083456:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.083462:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e056cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.083488:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e056cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760743.111598:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973194 is committed 00002000:00010000:1.0:1789760743.111718:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24064000 -> 24068096 00010000:00010000:1.0:1789760743.111733:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e056cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.111746:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e056cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.111756:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e056cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.111767:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760743.111772:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e056cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.111781:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e056cf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.111801:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e056cf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.111808:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e056cf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760743.111933:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493734656:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28559us (28788us total) trans 4294973194 rc 0/0 00000100:00100000:1.0:1789760743.111944:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5905 00000100:00100000:1.0:1789760743.111952:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753386238954 time 753424393158 state 0x2 ref 0 00000100:00100000:0.0:1789760743.118830:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760743.119014:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493734784 00000100:00100000:3.0:1789760743.119033:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493734784 00000100:00100000:3.0:1789760743.119087:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753386238954 time 753424393158 state 0x0 ref 1 00000100:00100000:3.0:1789760743.119103:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753424653574 time 753424393158 state 0x1 ref 1 00000100:00100000:3.0:1789760743.119127:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753424653574 time 753460163156 state 0x0 ref 1 00000100:00100000:3.0:1789760743.119140:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493734784:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760743.119197:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24068096, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:3.0:1789760743.119207:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e056d6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.119230:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:3.0:1789760743.119235:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.119241:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e056d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.119254:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e056d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.144766:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973195 is committed 00002000:00010000:3.0:1789760743.144829:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24068096 -> 24072192 00010000:00010000:3.0:1789760743.144845:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e056d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.144865:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e056d6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.144875:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e056d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.144883:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760743.144888:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e056d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.144897:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e056d6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.144918:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e056d6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.144925:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e056d6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.145054:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493734784:12345-192.168.203.60@tcp:4:dd.0 Request processed in 25910us (26227us total) trans 4294973195 rc 0/0 00000100:00100000:3.0:1789760743.145064:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5906 00000100:00100000:3.0:1789760743.145073:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753424653574 time 753460163156 state 0x2 ref 0 00000100:00100000:0.0:1789760743.150310:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760743.150413:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493734912 00000100:00100000:1.0:1789760743.150422:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493734912 00000100:00100000:1.0:1789760743.150484:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753424653574 time 753460163156 state 0x0 ref 1 00000100:00100000:1.0:1789760743.150498:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753460423572 time 753460163156 state 0x1 ref 1 00000100:00100000:1.0:1789760743.150583:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753460423572 time 753491617464 state 0x0 ref 1 00000100:00100000:1.0:1789760743.150599:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493734912:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760743.150645:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24072192, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:1.0:1789760743.150653:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e056dd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.150669:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760743.150673:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760743.150680:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e056dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.150693:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e056dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.186702:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973196 is committed 00002000:00010000:0.0:1789760743.186777:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24072192 -> 24076288 00010000:00010000:0.0:1789760743.186791:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e056dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.186804:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e056dd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.186814:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e056dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.186821:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760743.186826:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e056dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.186837:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e056dd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.186856:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e056dd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.186863:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e056dd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.187098:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493734912:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36495us (36792us total) trans 4294973196 rc 0/0 00000100:00100000:0.0:1789760743.187111:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5907 00000100:00100000:0.0:1789760743.187121:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753460423572 time 753491617464 state 0x2 ref 0 00000100:00100000:3.0:1789760743.190894:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760743.191050:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493735040 00000100:00100000:1.0:1789760743.191061:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493735040 00000100:00100000:1.0:1789760743.191116:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753460423572 time 753491617464 state 0x0 ref 1 00000100:00100000:1.0:1789760743.191131:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753491877880 time 753491617464 state 0x1 ref 1 00000100:00100000:1.0:1789760743.191160:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753491877880 time 753532195617 state 0x0 ref 1 00000100:00100000:1.0:1789760743.191173:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493735040:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760743.191218:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24076288, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:1.0:1789760743.191227:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e056e4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.191248:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760743.191252:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760743.191260:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e056e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.191273:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e056e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760743.223670:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973197 is committed 00002000:00010000:1.0:1789760743.224308:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24076288 -> 24080384 00010000:00010000:1.0:1789760743.224323:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e056e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.224336:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e056e4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.224345:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e056e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.224353:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760743.224358:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e056e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.224368:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e056e4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.224387:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e056e4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.224396:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e056e4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760743.224604:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493735040:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33428us (33713us total) trans 4294973197 rc 0/0 00000100:00100000:1.0:1789760743.224616:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5908 00000100:00100000:1.0:1789760743.224626:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753491877880 time 753532195617 state 0x2 ref 0 00000100:00100000:1.0:1789760743.229248:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760743.229342:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493735168 00000100:00100000:2.0:1789760743.229352:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493735168 00000100:00100000:2.0:1789760743.229405:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753491877880 time 753532195617 state 0x0 ref 1 00000100:00100000:2.0:1789760743.229420:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753532456033 time 753532195617 state 0x1 ref 1 00000100:00100000:2.0:1789760743.229448:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753532456033 time 753570484067 state 0x0 ref 1 00000100:00100000:2.0:1789760743.229460:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493735168:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760743.229525:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24080384, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:2.0:1789760743.229535:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e056eb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.229547:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760743.229550:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.229554:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e056eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.229564:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e056eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760743.260849:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973198 is committed 00002000:00010000:3.0:1789760743.260925:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24080384 -> 24084480 00010000:00010000:3.0:1789760743.260940:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e056eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.260991:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e056eb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.261001:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e056eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.261008:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760743.261013:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e056eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.261023:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e056eb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.261047:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e056eb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.261056:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e056eb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.261146:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493735168:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31681us (31902us total) trans 4294973198 rc 0/0 00000100:00100000:3.0:1789760743.261155:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5909 00000100:00100000:3.0:1789760743.261165:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753532456033 time 753570484067 state 0x2 ref 0 00000100:00100000:0.0:1789760743.272629:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760743.273827:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493735296 00000100:00100000:3.0:1789760743.273840:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493735296 00000100:00100000:3.0:1789760743.273903:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753532456033 time 753570484067 state 0x0 ref 1 00000100:00100000:3.0:1789760743.273919:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753570744483 time 753570484067 state 0x1 ref 1 00000100:00100000:3.0:1789760743.274275:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753570744483 time 753615307924 state 0x0 ref 1 00000100:00100000:2.0:1789760743.275522:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493735296:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760743.275587:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24084480, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:2.0:1789760743.275606:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e056f2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.275621:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760743.275626:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.275633:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e056f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.275648:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e056f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760743.290713:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760743.290735:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc2885180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3304:1876699519252864:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760743.290802:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760743.290896:0:6691:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519252864 00000100:00100000:0.0:1789760743.290907:0:6691:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519252864 00000100:00100000:0.0:1789760743.290991:0:6691:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 200 00000100:00100000:0.0:1789760743.291001:0:6691:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519252864:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760743.291024:0:6691:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528154624 pending 1703936 free 3834642432 avail 3816816640 00000100:00100000:0.0:1789760743.291067:0:6691:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519252864:12345-0@lo:13:osp-pre-0-0.0 Request processed in 67us (264us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760743.291074:0:6691:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 200 00000100:00080000:1.0:1789760743.291575:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760743.291628:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc2885180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3304:1876699519252864:0@lo:13:osp-pre-0-0.0 00000001:00080000:2.0:1789760743.319432:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973199 is committed 00002000:00010000:3.0:1789760743.319697:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24084480 -> 24088576 00010000:00010000:3.0:1789760743.319710:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e056f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.319723:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e056f2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.319732:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e056f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.319740:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760743.319745:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e056f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.319755:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e056f2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.319774:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e056f2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.319783:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e056f2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.319873:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493735296:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45580us (47249us total) trans 4294973199 rc 0/0 00000100:00100000:3.0:1789760743.319883:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5910 00000100:00100000:3.0:1789760743.319893:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753570744483 time 753615307924 state 0x2 ref 0 00000100:00100000:0.0:1789760743.323172:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760743.323277:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493735424 00000100:00100000:1.0:1789760743.323288:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493735424 00000100:00100000:1.0:1789760743.323341:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753570744483 time 753615307924 state 0x0 ref 1 00000100:00100000:1.0:1789760743.323357:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753615568340 time 753615307924 state 0x1 ref 1 00000100:00100000:1.0:1789760743.323418:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753615568340 time 753664453412 state 0x0 ref 1 00000100:00100000:1.0:1789760743.323431:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493735424:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760743.323488:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24088576, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:1.0:1789760743.323499:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e056f9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.323525:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760743.323530:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760743.323536:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e056f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.323549:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e056f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760743.350905:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973200 is committed 00002000:00010000:3.0:1789760743.351146:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24088576 -> 24092672 00010000:00010000:3.0:1789760743.351162:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e056f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.351175:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e056f9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.351184:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e056f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.351200:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760743.351205:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e056f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.351215:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e056f9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.351240:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e056f9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.351247:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e056f9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.351342:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493735424:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27908us (28174us total) trans 4294973200 rc 0/0 00000100:00100000:3.0:1789760743.351352:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5911 00000100:00100000:3.0:1789760743.351363:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753615568340 time 753664453412 state 0x2 ref 0 00000100:00100000:2.0:1789760743.358754:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760743.358916:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493735552 00000100:00100000:1.0:1789760743.358926:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493735552 00000100:00100000:1.0:1789760743.359030:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753615568340 time 753664453412 state 0x0 ref 1 00000100:00100000:1.0:1789760743.359049:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753664713828 time 753664453412 state 0x1 ref 1 00000100:00100000:1.0:1789760743.359080:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753664713828 time 753700114974 state 0x0 ref 1 00000100:00100000:1.0:1789760743.359093:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493735552:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760743.359421:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24092672, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:2.0:1789760743.359432:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05700 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.359449:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760743.359454:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.359460:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05700 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.359496:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05700 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760743.396617:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24092672 -> 24096768 00000001:00080000:3.0:1789760743.396633:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973201 is committed 00010000:00010000:0.0:1789760743.396640:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05700 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.396654:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05700 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.396663:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05700 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.396676:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760743.396682:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05700 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.396690:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05700 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.396709:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05700 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.396716:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05700 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.396822:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493735552:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37726us (38072us total) trans 4294973201 rc 0/0 00000100:00100000:0.0:1789760743.396833:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5912 00000100:00100000:0.0:1789760743.396843:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753664713828 time 753700114974 state 0x2 ref 0 00000100:00100000:0.0:1789760743.403579:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760743.403702:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493735680 00000100:00100000:3.0:1789760743.403713:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493735680 00000100:00100000:3.0:1789760743.403768:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753664713828 time 753700114974 state 0x0 ref 1 00000100:00100000:3.0:1789760743.403783:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753700375390 time 753700114974 state 0x1 ref 1 00000100:00100000:3.0:1789760743.403816:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753700375390 time 753744851507 state 0x0 ref 1 00000100:00100000:3.0:1789760743.403828:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493735680:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760743.403875:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24096768, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:3.0:1789760743.403884:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05707 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.403906:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:3.0:1789760743.403911:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.403917:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05707 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.403936:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05707 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.438349:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973202 is committed 00002000:00010000:3.0:1789760743.438753:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24096768 -> 24100864 00010000:00010000:3.0:1789760743.438772:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05707 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.438786:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05707 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.438795:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05707 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.438802:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760743.438807:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05707 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.438816:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05707 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.438843:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05707 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.438851:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05707 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.438996:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493735680:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35129us (35385us total) trans 4294973202 rc 0/0 00000100:00100000:3.0:1789760743.439008:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5913 00000100:00100000:3.0:1789760743.439018:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753700375390 time 753744851507 state 0x2 ref 0 00000100:00100000:3.0:1789760743.443679:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760743.443776:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493735808 00000100:00100000:2.0:1789760743.443788:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493735808 00000100:00100000:2.0:1789760743.443845:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753700375390 time 753744851507 state 0x0 ref 1 00000100:00100000:2.0:1789760743.443860:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753745111923 time 753744851507 state 0x1 ref 1 00000100:00100000:2.0:1789760743.443893:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753745111923 time 753784928261 state 0x0 ref 1 00000100:00100000:2.0:1789760743.443906:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493735808:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760743.444151:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24100864, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:2.0:1789760743.444161:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0570e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.444178:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760743.444183:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.444189:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0570e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.444203:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0570e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760743.488810:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973203 is committed 00002000:00010000:3.0:1789760743.488902:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24100864 -> 24104960 00010000:00010000:3.0:1789760743.488918:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0570e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.488931:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0570e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.488940:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0570e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.488948:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760743.488953:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0570e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.489689:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0570e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.489710:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0570e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.489719:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0570e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.489824:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493735808:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45915us (46151us total) trans 4294973203 rc 0/0 00000100:00100000:3.0:1789760743.489834:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5914 00000100:00100000:3.0:1789760743.489844:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753745111923 time 753784928261 state 0x2 ref 0 00000100:00100000:0.0:1789760743.492919:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760743.493042:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493735936 00000100:00100000:1.0:1789760743.493051:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493735936 00000100:00100000:1.0:1789760743.493104:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753745111923 time 753784928261 state 0x0 ref 1 00000100:00100000:1.0:1789760743.493120:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753785188677 time 753784928261 state 0x1 ref 1 00000100:00100000:1.0:1789760743.493238:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753785188677 time 753834272195 state 0x0 ref 1 00000100:00100000:1.0:1789760743.493255:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493735936:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760743.493300:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24104960, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:1.0:1789760743.493308:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05715 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.493322:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760743.493326:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760743.493332:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05715 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.493345:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05715 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760743.544751:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24104960 -> 24109056 00010000:00010000:3.0:1789760743.544771:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05715 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.544794:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05715 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.544803:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05715 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.544810:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760743.544815:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05715 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.544824:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05715 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.544843:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05715 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.544851:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05715 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.544945:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493735936:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51689us (52031us total) trans 4294973204 rc 0/0 00000100:00100000:3.0:1789760743.544956:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5915 00000100:00100000:3.0:1789760743.544994:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753785188677 time 753834272195 state 0x2 ref 0 00000001:00080000:0.0:1789760743.545946:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973204 is committed 00000100:00100000:0.0:1789760743.550058:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760743.550146:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493736064 00000100:00100000:2.0:1789760743.550157:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493736064 00000100:00100000:2.0:1789760743.550211:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753785188677 time 753834272195 state 0x0 ref 1 00000100:00100000:2.0:1789760743.550226:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753834532611 time 753834272195 state 0x1 ref 1 00000100:00100000:2.0:1789760743.550254:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753834532611 time 753891290696 state 0x0 ref 1 00000100:00100000:2.0:1789760743.550266:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493736064:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760743.550313:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24109056, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:2.0:1789760743.550322:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0571c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.550338:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760743.550342:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.550349:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0571c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.550362:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0571c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760743.590020:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973205 is committed 00002000:00010000:2.0:1789760743.590107:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24109056 -> 24113152 00010000:00010000:2.0:1789760743.590125:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0571c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.590140:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0571c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.590150:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0571c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.590157:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760743.590163:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0571c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.590172:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0571c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.590191:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0571c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.590199:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0571c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.590290:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493736064:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40022us (40238us total) trans 4294973205 rc 0/0 00000100:00100000:2.0:1789760743.590300:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5916 00000100:00100000:2.0:1789760743.590309:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753834532611 time 753891290696 state 0x2 ref 0 00000100:00100000:2.0:1789760743.593301:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760743.593402:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493736192 00000100:00100000:3.0:1789760743.593420:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493736192 00000100:00100000:3.0:1789760743.593632:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753834532611 time 753891290696 state 0x0 ref 1 00000100:00100000:3.0:1789760743.593653:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753891551112 time 753891290696 state 0x1 ref 1 00000100:00100000:3.0:1789760743.593755:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753891551112 time 753934789004 state 0x0 ref 1 00000100:00100000:3.0:1789760743.593838:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493736192:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760743.593909:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24113152, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xb807 00010000:00010000:3.0:1789760743.593923:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05723 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.593945:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:3.0:1789760743.594013:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.594022:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05723 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.594041:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05723 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760743.632239:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973206 is committed 00002000:00010000:2.0:1789760743.632355:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24113152 -> 24117248 00002000:00010000:2.0:1789760743.632361:0:13038:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760742 -> 1789760743 00002000:00010000:2.0:1789760743.632364:0:13038:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760742 -> 1789760743 00010000:00010000:2.0:1789760743.632379:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05723 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.632392:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05723 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.632402:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05723 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.632410:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760743.632415:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05723 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.632425:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05723 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.632448:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05723 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.632456:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05723 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.632770:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493736192:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38921us (39397us total) trans 4294973206 rc 0/0 00000100:00100000:2.0:1789760743.632784:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5917 00000100:00100000:2.0:1789760743.632793:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753891551112 time 753934789004 state 0x2 ref 0 00000100:00100000:2.0:1789760743.639147:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760743.639229:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493736320 00000100:00100000:3.0:1789760743.639239:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493736320 00000100:00100000:3.0:1789760743.639291:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753891551112 time 753934789004 state 0x0 ref 1 00000100:00100000:3.0:1789760743.639306:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753935049420 time 753934789004 state 0x1 ref 1 00000100:00100000:3.0:1789760743.639333:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753935049420 time 753980369643 state 0x0 ref 1 00000100:00100000:3.0:1789760743.639346:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493736320:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760743.639394:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24117248, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xb807 00010000:00010000:3.0:1789760743.639402:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0572a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.639417:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=753s v=(0 0) 00010000:00010000:3.0:1789760743.639421:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.639428:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0572a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.639443:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0572a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760743.694817:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973207 is committed 00002000:00010000:2.0:1789760743.697088:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24117248 -> 24121344 00002000:00010000:2.0:1789760743.697098:0:6694:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb blocks from disk: 47111 -> 49159 00010000:00010000:2.0:1789760743.697113:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0572a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.697126:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0572a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.697135:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0572a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.697143:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760743.697148:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0572a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.697157:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0572a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.697178:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0572a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.697186:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0572a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.697296:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493736320:12345-192.168.203.60@tcp:4:dd.0 Request processed in 57946us (58154us total) trans 4294973207 rc 0/0 00000100:00100000:2.0:1789760743.697307:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5918 00000100:00100000:2.0:1789760743.697317:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753935049420 time 753980369643 state 0x2 ref 0 00000100:00100000:0.0:1789760743.711887:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760743.712003:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493736448 00000100:00100000:1.0:1789760743.712014:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493736448 00000100:00100000:1.0:1789760743.712068:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753935049420 time 753980369643 state 0x0 ref 1 00000100:00100000:1.0:1789760743.712084:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753980630059 time 753980369643 state 0x1 ref 1 00000100:00100000:1.0:1789760743.712116:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753980630059 time 754053152215 state 0x0 ref 1 00000100:00100000:1.0:1789760743.712129:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493736448:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760743.712177:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24121344, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:1.0:1789760743.712187:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05731 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.712209:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760743.712213:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760743.712219:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05731 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.712234:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05731 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760743.751450:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24121344 -> 24125440 00010000:00010000:1.0:1789760743.751479:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05731 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.751492:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05731 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.751500:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05731 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.751509:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760743.751523:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05731 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.751533:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05731 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.751732:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05731 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.751742:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05731 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760743.751847:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493736448:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39714us (39966us total) trans 4294973208 rc 0/0 00000100:00100000:1.0:1789760743.751858:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5919 00000100:00100000:1.0:1789760743.751868:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753980630059 time 754053152215 state 0x2 ref 0 00000001:00080000:3.0:1789760743.752312:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973208 is committed 00000100:00100000:3.0:1789760743.759638:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760743.759794:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493736576 00000100:00100000:2.0:1789760743.759805:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493736576 00000100:00100000:2.0:1789760743.759863:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 753980630059 time 754053152215 state 0x0 ref 1 00000100:00100000:2.0:1789760743.759880:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754053412631 time 754053152215 state 0x1 ref 1 00000100:00100000:2.0:1789760743.760069:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754053412631 time 754101103300 state 0x0 ref 1 00000100:00100000:2.0:1789760743.760085:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493736576:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760743.760134:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24125440, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:2.0:1789760743.760144:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05738 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.760161:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:2.0:1789760743.760167:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.760174:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05738 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.760190:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05738 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.796141:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973209 is committed 00002000:00010000:0.0:1789760743.796383:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24125440 -> 24129536 00010000:00010000:0.0:1789760743.796398:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05738 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.796411:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05738 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.796421:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05738 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.796429:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760743.796434:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05738 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.796444:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05738 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.796474:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05738 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.796482:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05738 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.796637:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493736576:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36540us (36993us total) trans 4294973209 rc 0/0 00000100:00100000:0.0:1789760743.796647:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5920 00000100:00100000:0.0:1789760743.796658:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754053412631 time 754101103300 state 0x2 ref 0 00000100:00100000:1.0:1789760743.802372:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4d94000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3304:1876699519252992:0@lo:400:kworker.0 00000100:00100000:2.0:1789760743.802423:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4d95180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519253248:0@lo:400:kworker.0 00000100:00100000:0.0:1789760743.802486:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4d94e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3303:1876699519253376:0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.802645:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4d97800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3305:1876699519253120:0@lo:400:kworker.0 00000100:00100000:2.0:1789760743.804879:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760743.804881:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760743.804976:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760743.805017:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519253376 00000100:00100000:3.0:1789760743.805025:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519253376 00000100:00100000:3.0:1789760743.805078:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519253120 00000100:00100000:3.0:1789760743.805081:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519253120 00000100:00100000:3.0:1789760743.805105:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 484 00000100:00100000:3.0:1789760743.805113:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876699519253376:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.805168:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876699519253376:12345-0@lo:400:kworker.0 Request processed in 55us (290us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760743.805175:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 484 00000100:00100000:3.0:1789760743.805191:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 485 00000100:00100000:3.0:1789760743.805198:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3305:x1876699519253120:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.805230:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3305:x1876699519253120:12345-0@lo:400:kworker.0 Request processed in 33us (270us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760743.805237:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 485 00000100:00100000:1.0:1789760743.805324:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1789760743.805353:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760743.805612:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4d94e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3303:1876699519253376:0@lo:400:kworker.0 00000100:00100000:0.0:1789760743.805633:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760743.805642:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4d96a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3303:1876699519253504:0@lo:400:kworker.0 00000100:00100000:0.0:1789760743.805673:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760743.805722:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519253248 00000100:00100000:2.0:1789760743.805728:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519253248 00000100:00100000:2.0:1789760743.805768:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 174 00000100:00100000:2.0:1789760743.805776:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519253248:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760743.805820:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519253248:12345-0@lo:400:kworker.0 Request processed in 45us (948us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760743.805827:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 174 00000100:00100000:1.0:1789760743.805862:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4d97800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3305:1876699519253120:0@lo:400:kworker.0 00000100:00100000:1.0:1789760743.805871:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4d96300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519253632:0@lo:400:kworker.0 00000100:00100000:1.0:1789760743.805897:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760743.805936:0:10475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519253632 00000100:00100000:1.0:1789760743.805940:0:10475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519253632 00000100:00100000:0.0:1789760743.806026:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519253504 00000100:00100000:0.0:1789760743.806030:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519253504 00000100:00100000:0.0:1789760743.806054:0:9460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 486 00000100:00100000:0.0:1789760743.806060:0:9460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876699519253504:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760743.806096:0:9460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876699519253504:12345-0@lo:400:kworker.0 Request processed in 37us (422us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760743.806102:0:9460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 486 00000100:00100000:2.0:1789760743.806208:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4d95180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519253248:0@lo:400:kworker.0 00000100:00100000:3.0:1789760743.806362:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519252992 00000100:00100000:3.0:1789760743.806368:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519252992 00000100:00100000:3.0:1789760743.806411:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 400 00000100:00100000:3.0:1789760743.806418:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d4c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3304:x1876699519252992:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760743.807243:0:10475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 175 00000100:00100000:1.0:1789760743.807255:0:10475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519253632:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760743.807303:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760743.807311:0:10475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519253632:12345-0@lo:400:kworker.0 Request processed in 59us (1411us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760743.807320:0:10475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 175 00000100:00100000:2.0:1789760743.807338:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493736704 00000100:00080000:1.0:1789760743.807343:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760743.807345:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493736704 00000100:00080000:1.0:1789760743.807350:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789760743.807383:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4d96300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519253632:0@lo:400:kworker.0 00000100:00100000:2.0:1789760743.807390:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754053412631 time 754101103300 state 0x0 ref 1 00000100:00100000:2.0:1789760743.807406:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754101363716 time 754101103300 state 0x1 ref 1 00000100:00100000:2.0:1789760743.807432:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754101363716 time 754148467500 state 0x0 ref 1 00000100:00100000:2.0:1789760743.807444:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493736704:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760743.807498:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24129536, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:2.0:1789760743.807508:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0573f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.807524:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:2.0:1789760743.807529:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.807536:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0573f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.807552:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0573f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.807564:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d4c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3304:x1876699519252992:12345-0@lo:400:kworker.0 Request processed in 1144us (2238us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760743.807573:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 400 00000100:00100000:1.0:1789760743.807664:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4d94000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3304:1876699519252992:0@lo:400:kworker.0 00000100:00100000:0.0:1789760743.807844:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4d96a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3303:1876699519253504:0@lo:400:kworker.0 00002000:00010000:0.0:1789760743.835703:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24129536 -> 24133632 00010000:00010000:0.0:1789760743.835722:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0573f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.835736:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0573f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.835746:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0573f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.835754:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760743.835759:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0573f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.835768:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0573f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.835788:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0573f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.835795:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0573f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.835920:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493736704:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28472us (28614us total) trans 4294973210 rc 0/0 00000100:00100000:0.0:1789760743.835931:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5921 00000100:00100000:0.0:1789760743.835941:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754101363716 time 754148467500 state 0x2 ref 0 00000001:00080000:3.0:1789760743.836043:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973210 is committed 00000100:00100000:1.0:1789760743.845336:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760743.845454:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493736832 00000100:00100000:3.0:1789760743.845475:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493736832 00000100:00100000:3.0:1789760743.845543:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754101363716 time 754148467500 state 0x0 ref 1 00000100:00100000:3.0:1789760743.845558:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754148727916 time 754148467500 state 0x1 ref 1 00000100:00100000:3.0:1789760743.845587:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754148727916 time 754186622006 state 0x0 ref 1 00000100:00100000:3.0:1789760743.845600:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493736832:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760743.845647:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24133632, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:3.0:1789760743.845657:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05746 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.845673:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760743.845678:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.845684:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05746 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.845698:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05746 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760743.885692:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24133632 -> 24137728 00010000:00010000:3.0:1789760743.885712:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05746 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.885726:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05746 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.885735:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05746 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.885742:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760743.885746:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05746 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.885754:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05746 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.885772:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05746 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.885778:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05746 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.885917:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493736832:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40314us (40588us total) trans 4294973211 rc 0/0 00000100:00100000:3.0:1789760743.885927:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5922 00000100:00100000:3.0:1789760743.885937:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754148727916 time 754186622006 state 0x2 ref 0 00000001:00080000:0.0:1789760743.886031:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973211 is committed 00000100:00100000:2.0:1789760743.894479:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760743.894653:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493736960 00000100:00100000:1.0:1789760743.894664:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493736960 00000100:00100000:1.0:1789760743.894724:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754148727916 time 754186622006 state 0x0 ref 1 00000100:00100000:1.0:1789760743.894740:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754186882422 time 754186622006 state 0x1 ref 1 00000100:00100000:1.0:1789760743.894815:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754186882422 time 754235850060 state 0x0 ref 1 00000100:00100000:1.0:1789760743.894830:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493736960:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760743.894881:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24137728, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:1.0:1789760743.894892:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0574d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.894909:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760743.894921:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760743.894927:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0574d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.894941:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0574d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760743.943319:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24137728 -> 24141824 00010000:00010000:3.0:1789760743.943339:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0574d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.943353:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0574d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.943362:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0574d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.943370:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760743.943374:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0574d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.943384:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0574d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.943402:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0574d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.943411:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0574d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.943658:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493736960:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48827us (49449us total) trans 4294973212 rc 0/0 00000100:00100000:3.0:1789760743.943670:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5923 00000100:00100000:3.0:1789760743.943680:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754186882422 time 754235850060 state 0x2 ref 0 00000001:00080000:0.0:1789760743.943826:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973212 is committed 00000100:00100000:2.0:1789760743.948657:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760743.948749:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493737088 00000100:00100000:0.0:1789760743.948758:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493737088 00000100:00100000:0.0:1789760743.948817:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754186882422 time 754235850060 state 0x0 ref 1 00000100:00100000:0.0:1789760743.948833:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754236110476 time 754235850060 state 0x1 ref 1 00000100:00100000:0.0:1789760743.948862:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754236110476 time 754289897356 state 0x0 ref 1 00000100:00100000:0.0:1789760743.948875:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493737088:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760743.948924:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24141824, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:0.0:1789760743.948933:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05754 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.948988:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:0.0:1789760743.948993:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760743.948999:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05754 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.949012:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05754 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760743.980420:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973213 is committed 00002000:00010000:0.0:1789760743.980533:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24141824 -> 24145920 00010000:00010000:0.0:1789760743.980716:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05754 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.980729:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05754 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.980738:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05754 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.980745:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760743.980750:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05754 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.980759:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05754 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.980780:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05754 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.980788:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05754 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.981120:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493737088:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32241us (32467us total) trans 4294973213 rc 0/0 00000100:00100000:0.0:1789760743.981131:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5924 00000100:00100000:0.0:1789760743.981140:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754236110476 time 754289897356 state 0x2 ref 0 00000100:00100000:1.0:1789760743.985302:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760743.985390:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493737216 00000100:00100000:0.0:1789760743.985398:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493737216 00000100:00100000:0.0:1789760743.985442:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754236110476 time 754289897356 state 0x0 ref 1 00000100:00100000:0.0:1789760743.985453:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754290157772 time 754289897356 state 0x1 ref 1 00000100:00100000:0.0:1789760743.985489:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754290157772 time 754326523817 state 0x0 ref 1 00000100:00100000:0.0:1789760743.985501:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493737216:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760743.985680:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24145920, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:0.0:1789760743.985689:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0575b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.985704:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:0.0:1789760743.985709:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760743.985716:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0575b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.985730:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0575b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.994567:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760744.010720:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973214 is committed 00002000:00010000:2.0:1789760744.010724:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24145920 -> 24150016 00010000:00010000:2.0:1789760744.010739:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0575b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.010753:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0575b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.010761:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0575b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.010768:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760744.010771:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0575b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.010778:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0575b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.010794:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0575b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.010803:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0575b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.010920:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493737216:12345-192.168.203.60@tcp:4:dd.0 Request processed in 25415us (25623us total) trans 4294973214 rc 0/0 00000100:00100000:2.0:1789760744.010934:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5925 00000100:00100000:2.0:1789760744.010945:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754290157772 time 754326523817 state 0x2 ref 0 00000100:00100000:2.0:1789760744.017699:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760744.017783:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493737344 00000100:00100000:3.0:1789760744.017793:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493737344 00000100:00100000:3.0:1789760744.017846:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754290157772 time 754326523817 state 0x0 ref 1 00000100:00100000:3.0:1789760744.017862:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754326784233 time 754326523817 state 0x1 ref 1 00000100:00100000:3.0:1789760744.017891:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754326784233 time 754358926298 state 0x0 ref 1 00000100:00100000:3.0:1789760744.017904:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493737344:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760744.017951:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24150016, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:3.0:1789760744.017973:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05762 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.017989:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760744.017993:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.017999:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05762 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.018014:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05762 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.052014:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973215 is committed 00002000:00010000:3.0:1789760744.052148:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24150016 -> 24154112 00010000:00010000:3.0:1789760744.052163:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05762 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.052194:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05762 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.052202:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05762 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.052208:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760744.052213:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05762 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.052221:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05762 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.052239:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05762 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.052247:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05762 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.052341:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493737344:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34434us (34645us total) trans 4294973215 rc 0/0 00000100:00100000:3.0:1789760744.052351:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5926 00000100:00100000:3.0:1789760744.052361:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754326784233 time 754358926298 state 0x2 ref 0 00000100:00100000:0.0:1789760744.060772:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760744.061882:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493737472 00000100:00100000:3.0:1789760744.061893:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493737472 00000100:00100000:3.0:1789760744.062122:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754326784233 time 754358926298 state 0x0 ref 1 00000100:00100000:3.0:1789760744.062138:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754359186714 time 754358926298 state 0x1 ref 1 00000100:00100000:3.0:1789760744.062224:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754359186714 time 754403258929 state 0x0 ref 1 00000100:00100000:3.0:1789760744.062238:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493737472:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760744.062287:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24154112, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:3.0:1789760744.062296:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05769 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.062311:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760744.062315:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.062321:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05769 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.062334:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05769 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.116728:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973216 is committed 00002000:00010000:3.0:1789760744.116839:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24154112 -> 24158208 00010000:00010000:3.0:1789760744.116854:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05769 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.116867:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05769 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.116881:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05769 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.116887:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760744.116892:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05769 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.116901:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05769 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.116921:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05769 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.116929:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05769 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.117114:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493737472:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54873us (56344us total) trans 4294973216 rc 0/0 00000100:00100000:3.0:1789760744.117125:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5927 00000100:00100000:3.0:1789760744.117136:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754359186714 time 754403258929 state 0x2 ref 0 00000100:00100000:2.0:1789760744.123407:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760744.124674:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493737600 00000100:00100000:0.0:1789760744.124688:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493737600 00000100:00100000:1.0:1789760744.124830:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754359186714 time 754403258929 state 0x0 ref 1 00000100:00100000:1.0:1789760744.124847:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754403519345 time 754403258929 state 0x1 ref 1 00000100:00100000:1.0:1789760744.124933:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754403519345 time 754465968158 state 0x0 ref 1 00000100:00100000:1.0:1789760744.124948:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493737600:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760744.125026:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24158208, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:1.0:1789760744.125036:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05770 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.125051:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760744.125055:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760744.125062:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05770 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.125076:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05770 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760744.163701:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973217 is committed 00002000:00010000:1.0:1789760744.163864:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24158208 -> 24162304 00010000:00010000:1.0:1789760744.163877:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05770 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.163890:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05770 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.163903:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05770 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.163910:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760744.163915:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05770 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.163923:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05770 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.163943:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05770 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.163951:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05770 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760744.164065:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493737600:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39116us (40721us total) trans 4294973217 rc 0/0 00000100:00100000:1.0:1789760744.164075:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5928 00000100:00100000:1.0:1789760744.164085:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754403519345 time 754465968158 state 0x2 ref 0 00000100:00100000:0.0:1789760744.167581:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760744.167679:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493737728 00000100:00100000:2.0:1789760744.167689:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493737728 00000100:00100000:2.0:1789760744.167738:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754403519345 time 754465968158 state 0x0 ref 1 00000100:00100000:2.0:1789760744.167749:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754466228574 time 754465968158 state 0x1 ref 1 00000100:00100000:2.0:1789760744.167771:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754466228574 time 754508808150 state 0x0 ref 1 00000100:00100000:2.0:1789760744.167780:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493737728:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760744.167820:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24162304, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:2.0:1789760744.167827:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05777 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.167839:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:2.0:1789760744.167843:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.167847:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05777 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.167857:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05777 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760744.201182:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24162304 -> 24166400 00010000:00010000:0.0:1789760744.201203:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05777 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.201217:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05777 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.201226:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05777 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.201234:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760744.201238:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05777 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.201248:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05777 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.201274:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05777 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.201282:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05777 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.201370:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493737728:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33585us (33794us total) trans 4294973218 rc 0/0 00000100:00100000:0.0:1789760744.201380:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5929 00000100:00100000:0.0:1789760744.201391:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754466228574 time 754508808150 state 0x2 ref 0 00000001:00080000:1.0:1789760744.201737:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973218 is committed 00000100:00100000:0.0:1789760744.207828:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760744.208015:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493737856 00000100:00100000:3.0:1789760744.208025:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493737856 00000100:00100000:3.0:1789760744.208081:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754466228574 time 754508808150 state 0x0 ref 1 00000100:00100000:3.0:1789760744.208142:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754509068566 time 754508808150 state 0x1 ref 1 00000100:00100000:3.0:1789760744.208175:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754509068566 time 754549210708 state 0x0 ref 1 00000100:00100000:3.0:1789760744.208188:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493737856:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760744.208237:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24166400, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:3.0:1789760744.208247:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0577e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.208263:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760744.208267:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.208274:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0577e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.208288:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0577e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.241854:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973219 is committed 00002000:00010000:1.0:1789760744.242157:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24166400 -> 24170496 00010000:00010000:1.0:1789760744.242171:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0577e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.242188:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0577e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.242197:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0577e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.242203:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760744.242208:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0577e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.242218:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0577e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.242237:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0577e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.242244:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0577e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760744.242378:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493737856:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34187us (34556us total) trans 4294973219 rc 0/0 00000100:00100000:1.0:1789760744.242399:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5930 00000100:00100000:1.0:1789760744.242408:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754509068566 time 754549210708 state 0x2 ref 0 00000100:00100000:2.0:1789760744.248602:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760744.248686:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493737984 00000100:00100000:3.0:1789760744.248695:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493737984 00000100:00100000:3.0:1789760744.248745:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754509068566 time 754549210708 state 0x0 ref 1 00000100:00100000:3.0:1789760744.248759:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754549471124 time 754549210708 state 0x1 ref 1 00000100:00100000:3.0:1789760744.248786:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754549471124 time 754589821629 state 0x0 ref 1 00000100:00100000:3.0:1789760744.248797:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493737984:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760744.248842:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24170496, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:3.0:1789760744.248851:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05785 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.248866:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760744.248870:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.248876:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05785 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.248890:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05785 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.279405:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973220 is committed 00002000:00010000:0.0:1789760744.279498:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24170496 -> 24174592 00010000:00010000:0.0:1789760744.279579:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05785 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.279593:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05785 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.279602:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05785 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.279610:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760744.279615:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05785 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.279630:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05785 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.279650:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05785 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.279657:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05785 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.279818:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493737984:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31017us (31220us total) trans 4294973220 rc 0/0 00000100:00100000:0.0:1789760744.279829:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5931 00000100:00100000:0.0:1789760744.279838:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754549471124 time 754589821629 state 0x2 ref 0 00000100:00100000:0.0:1789760744.290841:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760744.292122:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493738112 00000100:00100000:1.0:1789760744.292136:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493738112 00000100:00100000:1.0:1789760744.292198:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754549471124 time 754589821629 state 0x0 ref 1 00000100:00100000:1.0:1789760744.292215:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754590082045 time 754589821629 state 0x1 ref 1 00000100:00100000:1.0:1789760744.292321:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754590082045 time 754633291153 state 0x0 ref 1 00000100:00100000:1.0:1789760744.292408:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493738112:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760744.292455:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24174592, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:1.0:1789760744.292476:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0578c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.292492:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760744.292496:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760744.292503:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0578c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.292517:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0578c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760744.314438:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760744.314471:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc2885f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519253760:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760744.314546:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760744.314612:0:6691:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519253760 00000100:00100000:1.0:1789760744.314619:0:6691:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519253760 00000100:00100000:1.0:1789760744.314666:0:6691:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 201 00000100:00100000:1.0:1789760744.314676:0:6691:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519253760:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760744.314711:0:6691:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760744.314757:0:6691:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519253760:12345-0@lo:13:osp-pre-1-0.0 Request processed in 82us (209us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760744.314764:0:6691:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 201 00000100:00100000:1.0:1789760744.314812:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc2885f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519253760:0@lo:13:osp-pre-1-0.0 00000001:00080000:0.0:1789760744.320493:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973221 is committed 00002000:00010000:1.0:1789760744.320822:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24174592 -> 24178688 00010000:00010000:1.0:1789760744.320838:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0578c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.320850:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0578c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.320858:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0578c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.320866:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760744.320871:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0578c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.320880:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0578c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.320901:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0578c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.320908:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0578c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760744.321283:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493738112:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28875us (30447us total) trans 4294973221 rc 0/0 00000100:00100000:1.0:1789760744.321295:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5932 00000100:00100000:1.0:1789760744.321305:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754590082045 time 754633291153 state 0x2 ref 0 00000100:00100000:2.0:1789760744.327905:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760744.329660:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493738240 00000100:00100000:0.0:1789760744.329673:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493738240 00000100:00100000:0.0:1789760744.329733:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754590082045 time 754633291153 state 0x0 ref 1 00000100:00100000:0.0:1789760744.329748:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754633551569 time 754633291153 state 0x1 ref 1 00000100:00100000:0.0:1789760744.329832:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754633551569 time 754670866673 state 0x0 ref 1 00000100:00100000:0.0:1789760744.329847:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493738240:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760744.329900:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24178688, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:0.0:1789760744.329908:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05793 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.329923:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:0.0:1789760744.329927:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.329933:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05793 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.329946:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05793 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760744.363221:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973222 is committed 00002000:00010000:1.0:1789760744.365770:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24178688 -> 24182784 00010000:00010000:1.0:1789760744.365790:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05793 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.365802:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05793 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.365810:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05793 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.365817:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760744.365822:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05793 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.365829:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05793 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.365848:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05793 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.365860:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05793 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760744.366018:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493738240:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36118us (38067us total) trans 4294973222 rc 0/0 00000100:00100000:1.0:1789760744.366027:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5933 00000100:00100000:1.0:1789760744.366038:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754633551569 time 754670866673 state 0x2 ref 0 00000100:00100000:2.0:1789760744.371715:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760744.371782:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493738368 00000100:00100000:3.0:1789760744.371792:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493738368 00000100:00100000:3.0:1789760744.371849:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754633551569 time 754670866673 state 0x0 ref 1 00000100:00100000:3.0:1789760744.371865:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754671127089 time 754670866673 state 0x1 ref 1 00000100:00100000:3.0:1789760744.371895:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754671127089 time 754712930950 state 0x0 ref 1 00000100:00100000:3.0:1789760744.371907:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493738368:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760744.371993:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24182784, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:3.0:1789760744.372003:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0579a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.372019:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760744.372024:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.372031:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0579a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.372047:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0579a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760744.414212:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973223 is committed 00002000:00010000:3.0:1789760744.414306:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24182784 -> 24186880 00010000:00010000:3.0:1789760744.414320:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0579a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.414333:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0579a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.414343:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0579a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.414350:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760744.414355:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0579a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.414365:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0579a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.414383:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0579a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.414390:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0579a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.414574:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493738368:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42664us (42865us total) trans 4294973223 rc 0/0 00000100:00100000:3.0:1789760744.414585:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5934 00000100:00100000:3.0:1789760744.414594:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754671127089 time 754712930950 state 0x2 ref 0 00000100:00100000:0.0:1789760744.418800:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760744.418878:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493738496 00000100:00100000:3.0:1789760744.418887:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493738496 00000100:00100000:3.0:1789760744.418948:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754671127089 time 754712930950 state 0x0 ref 1 00000100:00100000:3.0:1789760744.419002:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754713191366 time 754712930950 state 0x1 ref 1 00000100:00100000:3.0:1789760744.419031:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754713191366 time 754760066431 state 0x0 ref 1 00000100:00100000:3.0:1789760744.419044:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493738496:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760744.419089:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24186880, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:3.0:1789760744.419097:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e057a1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.419123:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760744.419127:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.419134:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e057a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.419148:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e057a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.454127:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973224 is committed 00002000:00010000:0.0:1789760744.454242:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24186880 -> 24190976 00010000:00010000:0.0:1789760744.454256:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e057a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.454271:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e057a1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.454280:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e057a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.454288:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760744.454293:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e057a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.454304:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e057a1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.454324:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e057a1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.454332:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e057a1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.454434:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493738496:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35387us (35637us total) trans 4294973224 rc 0/0 00000100:00100000:0.0:1789760744.454445:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5935 00000100:00100000:0.0:1789760744.454454:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754713191366 time 754760066431 state 0x2 ref 0 00000100:00100000:2.0:1789760744.458048:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760744.458142:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493738624 00000100:00100000:1.0:1789760744.458152:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493738624 00000100:00100000:1.0:1789760744.458209:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754713191366 time 754760066431 state 0x0 ref 1 00000100:00100000:1.0:1789760744.458224:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754760326847 time 754760066431 state 0x1 ref 1 00000100:00100000:1.0:1789760744.458254:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754760326847 time 754799289546 state 0x0 ref 1 00000100:00100000:1.0:1789760744.458266:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493738624:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760744.458316:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24190976, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:1.0:1789760744.458325:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e057a8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.458339:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760744.458344:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760744.458349:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e057a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.458367:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e057a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.492702:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973225 is committed 00002000:00010000:3.0:1789760744.492924:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24190976 -> 24195072 00010000:00010000:3.0:1789760744.492947:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e057a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.492976:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e057a8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.492986:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e057a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.492993:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760744.492998:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e057a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.493007:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e057a8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.493028:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e057a8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.493036:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e057a8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.493158:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493738624:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34889us (35114us total) trans 4294973225 rc 0/0 00000100:00100000:3.0:1789760744.493172:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5936 00000100:00100000:3.0:1789760744.493182:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754760326847 time 754799289546 state 0x2 ref 0 00000100:00100000:2.0:1789760744.497298:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760744.497392:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493738752 00000100:00100000:1.0:1789760744.497402:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493738752 00000100:00100000:1.0:1789760744.497583:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754760326847 time 754799289546 state 0x0 ref 1 00000100:00100000:1.0:1789760744.497597:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754799549962 time 754799289546 state 0x1 ref 1 00000100:00100000:1.0:1789760744.497624:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754799549962 time 754838660282 state 0x0 ref 1 00000100:00100000:1.0:1789760744.497636:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493738752:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760744.497681:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24195072, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:1.0:1789760744.497689:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e057af lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.497703:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760744.497708:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760744.497713:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e057af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.497726:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e057af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760744.535235:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24195072 -> 24199168 00010000:00010000:0.0:1789760744.535255:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e057af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.535266:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e057af lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.535274:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e057af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.535281:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760744.535286:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e057af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.535296:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e057af lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.535316:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e057af lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.535323:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e057af lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.539188:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493738752:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37773us (38119us total) trans 4294973226 rc 0/0 00000100:00100000:0.0:1789760744.539206:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5937 00000100:00100000:0.0:1789760744.539218:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754799549962 time 754838660282 state 0x2 ref 0 00000001:00080000:3.0:1789760744.539441:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973226 is committed 00000100:00100000:0.0:1789760744.539659:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760744.539766:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493738880 00000100:00100000:3.0:1789760744.539850:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493738880 00000100:00100000:3.0:1789760744.539897:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754799549962 time 754838660282 state 0x0 ref 1 00000100:00100000:3.0:1789760744.539910:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754838920698 time 754838660282 state 0x1 ref 1 00000100:00100000:3.0:1789760744.540096:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754838920698 time 754881130025 state 0x0 ref 1 00000100:00100000:3.0:1789760744.540110:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493738880:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760744.540159:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24199168, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:3.0:1789760744.540167:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e057b6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.540184:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760744.540189:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.540195:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e057b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.540208:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e057b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.576526:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973227 is committed 00002000:00010000:0.0:1789760744.578038:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24199168 -> 24203264 00010000:00010000:0.0:1789760744.578058:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e057b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.578071:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e057b6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.578080:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e057b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.578092:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760744.578097:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e057b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.578105:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e057b6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.578124:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e057b6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.578131:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e057b6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.578397:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493738880:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38284us (38734us total) trans 4294973227 rc 0/0 00000100:00100000:0.0:1789760744.578417:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5938 00000100:00100000:0.0:1789760744.578427:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754838920698 time 754881130025 state 0x2 ref 0 00000100:00100000:2.0:1789760744.583161:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760744.584538:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493739008 00000100:00100000:0.0:1789760744.584550:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493739008 00000100:00100000:0.0:1789760744.584609:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754838920698 time 754881130025 state 0x0 ref 1 00000100:00100000:0.0:1789760744.584626:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754881390441 time 754881130025 state 0x1 ref 1 00000100:00100000:0.0:1789760744.584649:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754881390441 time 754925685749 state 0x0 ref 1 00000100:00100000:0.0:1789760744.584664:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493739008:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760744.584710:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24203264, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xc007 00010000:00010000:0.0:1789760744.584719:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e057bd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.584734:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:0.0:1789760744.584738:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.584744:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e057bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.584757:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e057bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760744.617374:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24203264 -> 24207360 00002000:00010000:0.0:1789760744.617383:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760743 -> 1789760744 00002000:00010000:0.0:1789760744.617386:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760743 -> 1789760744 00010000:00010000:0.0:1789760744.617399:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e057bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.617422:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e057bd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.617429:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e057bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.617436:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760744.617440:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e057bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.617448:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e057bd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.617475:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e057bd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.617482:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e057bd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.617583:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493739008:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32916us (34427us total) trans 4294973228 rc 0/0 00000100:00100000:0.0:1789760744.617594:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5939 00000100:00100000:0.0:1789760744.617603:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754881390441 time 754925685749 state 0x2 ref 0 00000001:00080000:1.0:1789760744.618440:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973228 is committed 00000100:00100000:0.0:1789760744.628951:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760744.629291:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493739136 00000100:00100000:2.0:1789760744.629302:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493739136 00000100:00100000:2.0:1789760744.629361:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754881390441 time 754925685749 state 0x0 ref 1 00000100:00100000:2.0:1789760744.629376:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754925946165 time 754925685749 state 0x1 ref 1 00000100:00100000:2.0:1789760744.629405:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754925946165 time 754970440333 state 0x0 ref 1 00000100:00100000:2.0:1789760744.629417:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493739136:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760744.629477:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24207360, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:2.0:1789760744.629486:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e057c4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.629501:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=754s v=(0 0) 00010000:00010000:2.0:1789760744.629505:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.629510:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e057c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.629524:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e057c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.661728:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973229 is committed 00002000:00010000:2.0:1789760744.661833:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24207360 -> 24211456 00010000:00010000:2.0:1789760744.661849:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e057c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.661862:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e057c4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.661871:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e057c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.661879:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760744.661884:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e057c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.661894:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e057c4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.661922:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e057c4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.661930:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e057c4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.662478:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493739136:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33040us (33514us total) trans 4294973229 rc 0/0 00000100:00100000:2.0:1789760744.662488:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5940 00000100:00100000:2.0:1789760744.662498:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754925946165 time 754970440333 state 0x2 ref 0 00000100:00100000:2.0:1789760744.668259:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760744.668354:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493739264 00000100:00100000:1.0:1789760744.668363:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493739264 00000100:00100000:1.0:1789760744.668419:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754925946165 time 754970440333 state 0x0 ref 1 00000100:00100000:1.0:1789760744.668432:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754970700749 time 754970440333 state 0x1 ref 1 00000100:00100000:1.0:1789760744.668466:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754970700749 time 755009488974 state 0x0 ref 1 00000100:00100000:1.0:1789760744.668484:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493739264:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760744.668532:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24211456, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:1.0:1789760744.668610:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e057cb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.668627:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760744.668632:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760744.668639:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e057cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.668654:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e057cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760744.751334:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24211456 -> 24215552 00010000:00010000:1.0:1789760744.751355:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e057cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.751367:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e057cb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.751376:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e057cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.751384:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760744.751389:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e057cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.751398:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e057cb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.751416:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e057cb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.751424:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e057cb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760744.751590:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493739264:12345-192.168.203.60@tcp:4:dd.0 Request processed in 83107us (83333us total) trans 4294973230 rc 0/0 00000100:00100000:1.0:1789760744.751600:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5941 00000100:00100000:1.0:1789760744.751610:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754970700749 time 755009488974 state 0x2 ref 0 00000001:00080000:3.0:1789760744.752220:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973230 is committed 00000100:00100000:1.0:1789760744.757900:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760744.759834:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493739392 00000100:00100000:0.0:1789760744.759848:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493739392 00000100:00100000:0.0:1789760744.759903:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 754970700749 time 755009488974 state 0x0 ref 1 00000100:00100000:0.0:1789760744.759918:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755009749390 time 755009488974 state 0x1 ref 1 00000100:00100000:0.0:1789760744.760018:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755009749390 time 755101053534 state 0x0 ref 1 00000100:00100000:0.0:1789760744.760030:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493739392:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760744.760078:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24215552, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:0.0:1789760744.760086:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e057d2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.760101:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760744.760105:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.760111:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e057d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.760125:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e057d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.816292:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973231 is committed 00002000:00010000:3.0:1789760744.816393:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24215552 -> 24219648 00010000:00010000:3.0:1789760744.816409:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e057d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.816420:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e057d2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.816428:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e057d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.816435:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760744.816440:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e057d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.816448:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e057d2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.816478:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e057d2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.816487:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e057d2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.816586:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493739392:12345-192.168.203.60@tcp:4:dd.0 Request processed in 56553us (58692us total) trans 4294973231 rc 0/0 00000100:00100000:3.0:1789760744.816596:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5942 00000100:00100000:3.0:1789760744.816606:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755009749390 time 755101053534 state 0x2 ref 0 00000100:00100000:2.0:1789760744.825303:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760744.825592:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493739520 00000100:00100000:0.0:1789760744.825603:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493739520 00000100:00100000:0.0:1789760744.825658:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755009749390 time 755101053534 state 0x0 ref 1 00000100:00100000:0.0:1789760744.825740:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755101313950 time 755101053534 state 0x1 ref 1 00000100:00100000:0.0:1789760744.825824:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755101313950 time 755166859897 state 0x0 ref 1 00000100:00100000:0.0:1789760744.825837:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493739520:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760744.825888:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24219648, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:0.0:1789760744.825897:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e057d9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.825922:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760744.825927:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.825934:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e057d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.825948:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e057d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.856020:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973232 is committed 00002000:00010000:0.0:1789760744.856103:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24219648 -> 24223744 00010000:00010000:0.0:1789760744.856118:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e057d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.856131:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e057d9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.856139:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e057d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.856146:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760744.856151:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e057d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.856161:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e057d9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.856179:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e057d9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.856187:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e057d9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.856272:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493739520:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30433us (30975us total) trans 4294973232 rc 0/0 00000100:00100000:0.0:1789760744.856283:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5943 00000100:00100000:0.0:1789760744.856292:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755101313950 time 755166859897 state 0x2 ref 0 00000100:00100000:0.0:1789760744.859152:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760744.859276:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493739648 00000100:00100000:2.0:1789760744.859284:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493739648 00000100:00100000:2.0:1789760744.859336:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755101313950 time 755166859897 state 0x0 ref 1 00000100:00100000:2.0:1789760744.859351:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755167120313 time 755166859897 state 0x1 ref 1 00000100:00100000:2.0:1789760744.859379:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755167120313 time 755200415053 state 0x0 ref 1 00000100:00100000:2.0:1789760744.859391:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493739648:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760744.859437:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24223744, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:2.0:1789760744.859447:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e057e0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.859480:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760744.859484:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.859490:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e057e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.859513:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e057e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760744.891628:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973233 is committed 00002000:00010000:1.0:1789760744.891930:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24223744 -> 24227840 00010000:00010000:1.0:1789760744.891945:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e057e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.892017:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e057e0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.892029:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e057e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.892036:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760744.892042:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e057e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.892051:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e057e0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.892071:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e057e0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.892079:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e057e0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760744.892170:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493739648:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32775us (33022us total) trans 4294973233 rc 0/0 00000100:00100000:1.0:1789760744.892181:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5944 00000100:00100000:1.0:1789760744.892190:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755167120313 time 755200415053 state 0x2 ref 0 00000100:00100000:1.0:1789760744.898181:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760744.898346:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493739776 00000100:00100000:0.0:1789760744.898357:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493739776 00000100:00100000:0.0:1789760744.898408:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755167120313 time 755200415053 state 0x0 ref 1 00000100:00100000:0.0:1789760744.898423:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755200675469 time 755200415053 state 0x1 ref 1 00000100:00100000:0.0:1789760744.898464:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755200675469 time 755239488789 state 0x0 ref 1 00000100:00100000:0.0:1789760744.898477:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493739776:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760744.898523:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24227840, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:0.0:1789760744.898532:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e057e7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.898547:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760744.898551:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.898558:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e057e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.898572:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e057e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.958428:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973234 is committed 00002000:00010000:0.0:1789760744.958435:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24227840 -> 24231936 00010000:00010000:0.0:1789760744.958450:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e057e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.958476:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e057e7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.958485:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e057e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.958496:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760744.958501:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e057e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.958510:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e057e7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.958529:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e057e7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.958537:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e057e7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.958633:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493739776:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60152us (60456us total) trans 4294973234 rc 0/0 00000100:00100000:0.0:1789760744.958647:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5945 00000100:00100000:0.0:1789760744.958657:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755200675469 time 755239488789 state 0x2 ref 0 00000100:00100000:2.0:1789760744.962244:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760744.962406:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493739904 00000100:00100000:3.0:1789760744.962415:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493739904 00000100:00100000:3.0:1789760744.962476:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755200675469 time 755239488789 state 0x0 ref 1 00000100:00100000:3.0:1789760744.962491:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755239749205 time 755239488789 state 0x1 ref 1 00000100:00100000:3.0:1789760744.962585:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755239749205 time 755303619304 state 0x0 ref 1 00000100:00100000:3.0:1789760744.962599:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493739904:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760744.962650:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24231936, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:3.0:1789760744.962660:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e057ee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.962676:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760744.962680:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.962686:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e057ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.962700:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e057ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760745.009769:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973235 is committed 00002000:00010000:3.0:1789760745.010017:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24231936 -> 24236032 00010000:00010000:3.0:1789760745.010033:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e057ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.010050:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e057ee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.010059:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e057ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.010066:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760745.010070:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e057ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.010079:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e057ee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.010099:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e057ee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.010119:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e057ee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.010211:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493739904:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47609us (47972us total) trans 4294973235 rc 0/0 00000100:00100000:3.0:1789760745.010221:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5946 00000100:00100000:3.0:1789760745.010232:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755239749205 time 755303619304 state 0x2 ref 0 00000100:00100000:2.0:1789760745.014085:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760745.014259:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493740032 00000100:00100000:3.0:1789760745.014270:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493740032 00000100:00100000:3.0:1789760745.014342:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755239749205 time 755303619304 state 0x0 ref 1 00000100:00100000:3.0:1789760745.014357:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755303879720 time 755303619304 state 0x1 ref 1 00000100:00100000:3.0:1789760745.014378:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755303879720 time 755355414498 state 0x0 ref 1 00000100:00100000:3.0:1789760745.014391:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493740032:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760745.014440:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24236032, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:3.0:1789760745.014461:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e057f5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.014476:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760745.014480:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.014486:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e057f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.014498:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e057f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760745.014724:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789760745.021347:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760745.073158:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973236 is committed 00002000:00010000:2.0:1789760745.073261:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24236032 -> 24240128 00010000:00010000:2.0:1789760745.073278:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e057f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.073293:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e057f5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.073303:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e057f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.073311:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760745.073316:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e057f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.073326:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e057f5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.073346:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e057f5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.073355:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e057f5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760745.073486:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493740032:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59094us (59407us total) trans 4294973236 rc 0/0 00000100:00100000:2.0:1789760745.073497:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5947 00000100:00100000:2.0:1789760745.073570:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755303879720 time 755355414498 state 0x2 ref 0 00000100:00100000:2.0:1789760745.076668:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760745.076818:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493740160 00000100:00100000:1.0:1789760745.076828:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493740160 00000100:00100000:1.0:1789760745.076877:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755303879720 time 755355414498 state 0x0 ref 1 00000100:00100000:1.0:1789760745.076891:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755355674914 time 755355414498 state 0x1 ref 1 00000100:00100000:1.0:1789760745.076957:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755355674914 time 755417992090 state 0x0 ref 1 00000100:00100000:1.0:1789760745.077011:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493740160:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760745.077057:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24240128, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:1.0:1789760745.077066:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e057fc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.077081:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760745.077085:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760745.077091:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e057fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.077105:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e057fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760745.121776:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973237 is committed 00002000:00010000:1.0:1789760745.121852:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24240128 -> 24244224 00010000:00010000:1.0:1789760745.121868:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e057fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.121881:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e057fc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.121889:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e057fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.121895:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760745.121900:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e057fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.121908:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e057fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.121925:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e057fc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.121943:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e057fc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.122054:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493740160:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45042us (45391us total) trans 4294973237 rc 0/0 00000100:00100000:1.0:1789760745.122063:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5948 00000100:00100000:1.0:1789760745.122073:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755355674914 time 755417992090 state 0x2 ref 0 00000100:00100000:1.0:1789760745.127071:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760745.127149:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493740288 00000100:00100000:0.0:1789760745.127159:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493740288 00000100:00100000:0.0:1789760745.127210:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755355674914 time 755417992090 state 0x0 ref 1 00000100:00100000:0.0:1789760745.127225:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755418252506 time 755417992090 state 0x1 ref 1 00000100:00100000:0.0:1789760745.127257:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755418252506 time 755468292255 state 0x0 ref 1 00000100:00100000:0.0:1789760745.127269:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493740288:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760745.127316:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24244224, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:0.0:1789760745.127326:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05803 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.127341:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760745.127346:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.127352:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05803 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.127367:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05803 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760745.166418:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973238 is committed 00002000:00010000:0.0:1789760745.166631:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24244224 -> 24248320 00010000:00010000:0.0:1789760745.166647:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05803 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.166661:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05803 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.166670:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05803 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.166678:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760745.166683:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05803 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.166691:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05803 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.166710:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05803 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.166718:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05803 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760745.166848:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493740288:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39575us (39781us total) trans 4294973238 rc 0/0 00000100:00100000:0.0:1789760745.166858:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5949 00000100:00100000:0.0:1789760745.166868:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755418252506 time 755468292255 state 0x2 ref 0 00000100:00100000:0.0:1789760745.169411:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760745.169485:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493740416 00000100:00100000:3.0:1789760745.169494:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493740416 00000100:00100000:3.0:1789760745.169548:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755418252506 time 755468292255 state 0x0 ref 1 00000100:00100000:3.0:1789760745.169564:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755468552671 time 755468292255 state 0x1 ref 1 00000100:00100000:3.0:1789760745.169600:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755468552671 time 755510635015 state 0x0 ref 1 00000100:00100000:3.0:1789760745.169616:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493740416:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760745.169667:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24248320, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:3.0:1789760745.169675:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0580a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.169690:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760745.169695:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.169701:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0580a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.170103:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0580a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760745.203250:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973239 is committed 00002000:00010000:2.0:1789760745.203362:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24248320 -> 24252416 00010000:00010000:2.0:1789760745.203378:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0580a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.203390:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0580a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.203399:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0580a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.203407:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760745.203412:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0580a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.203422:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0580a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.203443:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0580a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.203468:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0580a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760745.203811:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493740416:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34192us (34401us total) trans 4294973239 rc 0/0 00000100:00100000:2.0:1789760745.203823:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5950 00000100:00100000:2.0:1789760745.203834:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755468552671 time 755510635015 state 0x2 ref 0 00000100:00100000:2.0:1789760745.207702:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760745.207845:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493740544 00000100:00100000:0.0:1789760745.207855:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493740544 00000100:00100000:0.0:1789760745.207909:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755468552671 time 755510635015 state 0x0 ref 1 00000100:00100000:0.0:1789760745.207924:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755510895431 time 755510635015 state 0x1 ref 1 00000100:00100000:0.0:1789760745.207957:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755510895431 time 755548992623 state 0x0 ref 1 00000100:00100000:0.0:1789760745.208004:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493740544:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760745.208050:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24252416, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:0.0:1789760745.208059:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05811 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.208074:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760745.208079:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.208085:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05811 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.208099:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05811 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760745.248810:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24252416 -> 24256512 00010000:00010000:0.0:1789760745.248829:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05811 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.248848:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05811 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.248856:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05811 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.248864:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760745.248869:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05811 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.248879:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05811 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.248898:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05811 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.248919:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05811 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760745.249043:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493740544:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41036us (41346us total) trans 4294973240 rc 0/0 00000100:00100000:0.0:1789760745.249054:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5951 00000100:00100000:0.0:1789760745.249063:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755510895431 time 755548992623 state 0x2 ref 0 00000001:00080000:3.0:1789760745.249520:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973240 is committed 00000100:00100000:1.0:1789760745.253292:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760745.253411:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493740672 00000100:00100000:2.0:1789760745.253422:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493740672 00000100:00100000:2.0:1789760745.253487:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755510895431 time 755548992623 state 0x0 ref 1 00000100:00100000:2.0:1789760745.254337:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755549253039 time 755548992623 state 0x1 ref 1 00000100:00100000:2.0:1789760745.254392:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755549253039 time 755595427033 state 0x0 ref 1 00000100:00100000:2.0:1789760745.254411:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3661500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493740672:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760745.254490:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24256512, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:2.0:1789760745.254500:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05818 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.254517:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760745.254522:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760745.254528:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05818 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.255302:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05818 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760745.290806:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973241 is committed 00002000:00010000:1.0:1789760745.290904:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24256512 -> 24260608 00010000:00010000:1.0:1789760745.290919:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05818 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.290933:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05818 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.290942:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05818 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.291104:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760745.291111:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05818 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.291123:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05818 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.291143:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05818 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.291151:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05818 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.291242:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3661500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493740672:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36832us (37952us total) trans 4294973241 rc 0/0 00000100:00100000:1.0:1789760745.291257:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5952 00000100:00100000:1.0:1789760745.291266:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755549253039 time 755595427033 state 0x2 ref 0 00000100:00100000:0.0:1789760745.295489:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760745.295562:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493740800 00000100:00100000:1.0:1789760745.295571:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493740800 00000100:00100000:1.0:1789760745.295627:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755549253039 time 755595427033 state 0x0 ref 1 00000100:00100000:1.0:1789760745.295642:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755595687449 time 755595427033 state 0x1 ref 1 00000100:00100000:1.0:1789760745.295670:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755595687449 time 755636706347 state 0x0 ref 1 00000100:00100000:1.0:1789760745.295682:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493740800:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760745.295730:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24260608, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:1.0:1789760745.295739:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0581f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.295754:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760745.295759:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760745.295765:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0581f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.295779:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0581f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760745.319474:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973242 is committed 00002000:00010000:1.0:1789760745.319626:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24260608 -> 24264704 00010000:00010000:1.0:1789760745.319641:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0581f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.319655:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0581f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.319665:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0581f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.319672:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760745.319677:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0581f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.319687:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0581f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.319706:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0581f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.319714:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0581f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.319806:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493740800:12345-192.168.203.60@tcp:4:dd.0 Request processed in 24121us (24322us total) trans 4294973242 rc 0/0 00000100:00100000:1.0:1789760745.319816:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5953 00000100:00100000:1.0:1789760745.319826:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755595687449 time 755636706347 state 0x2 ref 0 00000100:00100000:3.0:1789760745.323295:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760745.323491:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493740928 00000100:00100000:2.0:1789760745.323508:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493740928 00000100:00100000:2.0:1789760745.323565:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755595687449 time 755636706347 state 0x0 ref 1 00000100:00100000:2.0:1789760745.323580:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755636966763 time 755636706347 state 0x1 ref 1 00000100:00100000:2.0:1789760745.323614:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755636966763 time 755664649442 state 0x0 ref 1 00000100:00100000:2.0:1789760745.323627:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb05c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493740928:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760745.323677:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24264704, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:2.0:1789760745.323685:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05826 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.323700:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760745.323705:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760745.323711:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05826 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.323725:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05826 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760745.350768:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973243 is committed 00002000:00010000:0.0:1789760745.350846:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24264704 -> 24268800 00010000:00010000:0.0:1789760745.350861:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05826 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.350875:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05826 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.350885:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05826 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.350892:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760745.350897:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05826 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.350906:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05826 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.350924:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05826 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.351066:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05826 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760745.351186:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb05c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493740928:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27556us (27895us total) trans 4294973243 rc 0/0 00000100:00100000:0.0:1789760745.351196:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5954 00000100:00100000:0.0:1789760745.351206:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755636966763 time 755664649442 state 0x2 ref 0 00000100:00100000:1.0:1789760745.354390:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760745.354460:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493741056 00000100:00100000:0.0:1789760745.354469:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493741056 00000100:00100000:0.0:1789760745.354528:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755636966763 time 755664649442 state 0x0 ref 1 00000100:00100000:0.0:1789760745.354544:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755664909858 time 755664649442 state 0x1 ref 1 00000100:00100000:0.0:1789760745.354573:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755664909858 time 755695608017 state 0x0 ref 1 00000100:00100000:0.0:1789760745.354585:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3662680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493741056:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760745.354633:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24268800, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:0.0:1789760745.354641:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0582d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.354656:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760745.354660:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.354667:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0582d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.354680:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0582d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760745.387358:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973244 is committed 00002000:00010000:0.0:1789760745.387479:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24268800 -> 24272896 00010000:00010000:0.0:1789760745.387494:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0582d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.387614:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0582d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.387624:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0582d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.387632:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760745.387637:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0582d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.387646:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0582d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.387666:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0582d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.387674:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0582d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760745.387763:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3662680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493741056:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33175us (33375us total) trans 4294973244 rc 0/0 00000100:00100000:0.0:1789760745.387772:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5955 00000100:00100000:0.0:1789760745.387781:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755664909858 time 755695608017 state 0x2 ref 0 00000100:00100000:0.0:1789760745.392067:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760745.392274:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493741184 00000100:00100000:3.0:1789760745.392284:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493741184 00000100:00100000:3.0:1789760745.392339:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755664909858 time 755695608017 state 0x0 ref 1 00000100:00100000:3.0:1789760745.392355:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755695868433 time 755695608017 state 0x1 ref 1 00000100:00100000:3.0:1789760745.392384:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755695868433 time 755733419676 state 0x0 ref 1 00000100:00100000:3.0:1789760745.392397:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493741184:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760745.392467:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24272896, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:3.0:1789760745.392476:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05834 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.392491:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760745.392495:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.392692:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05834 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.392711:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05834 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760745.427090:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973245 is committed 00002000:00010000:3.0:1789760745.427348:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24272896 -> 24276992 00010000:00010000:3.0:1789760745.427364:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05834 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.427377:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05834 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.427386:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05834 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.427394:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760745.427399:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05834 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.427408:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05834 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.427432:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05834 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.427439:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05834 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.427599:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493741184:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35198us (35534us total) trans 4294973245 rc 0/0 00000100:00100000:3.0:1789760745.427611:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5956 00000100:00100000:3.0:1789760745.427620:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755695868433 time 755733419676 state 0x2 ref 0 00000100:00100000:2.0:1789760745.430739:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760745.430838:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493741312 00000100:00100000:1.0:1789760745.430848:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493741312 00000100:00100000:1.0:1789760745.430899:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755695868433 time 755733419676 state 0x0 ref 1 00000100:00100000:1.0:1789760745.430914:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755733680092 time 755733419676 state 0x1 ref 1 00000100:00100000:1.0:1789760745.431097:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755733680092 time 755772130870 state 0x0 ref 1 00000100:00100000:1.0:1789760745.431111:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493741312:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760745.431162:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24276992, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:1.0:1789760745.431171:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0583b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.431187:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760745.431192:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760745.431199:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0583b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.431213:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0583b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760745.466943:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973246 is committed 00002000:00010000:1.0:1789760745.467083:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24276992 -> 24281088 00010000:00010000:1.0:1789760745.467098:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0583b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.467115:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0583b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.467124:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0583b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.467130:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760745.467134:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0583b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.467142:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0583b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.467160:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0583b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.467167:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0583b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.467260:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493741312:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36145us (36524us total) trans 4294973246 rc 0/0 00000100:00100000:1.0:1789760745.467271:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5957 00000100:00100000:1.0:1789760745.467281:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755733680092 time 755772130870 state 0x2 ref 0 00000100:00100000:1.0:1789760745.470903:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760745.471175:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493741440 00000100:00100000:0.0:1789760745.471186:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493741440 00000100:00100000:0.0:1789760745.471307:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755733680092 time 755772130870 state 0x0 ref 1 00000100:00100000:0.0:1789760745.471323:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755772391286 time 755772130870 state 0x1 ref 1 00000100:00100000:0.0:1789760745.471345:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755772391286 time 755812382035 state 0x0 ref 1 00000100:00100000:0.0:1789760745.471356:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3662300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493741440:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760745.471405:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24281088, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:0.0:1789760745.471413:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05842 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.471429:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760745.471433:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.471439:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05842 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.471463:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05842 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760745.506023:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973247 is committed 00002000:00010000:0.0:1789760745.506228:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24281088 -> 24285184 00010000:00010000:0.0:1789760745.506245:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05842 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.506259:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05842 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.506275:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05842 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.506283:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760745.506288:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05842 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.506298:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05842 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.506319:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05842 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.506327:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05842 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760745.506422:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3662300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493741440:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35061us (35523us total) trans 4294973247 rc 0/0 00000100:00100000:0.0:1789760745.506433:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5958 00000100:00100000:0.0:1789760745.506442:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755772391286 time 755812382035 state 0x2 ref 0 00000100:00100000:3.0:1789760745.509294:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760745.509664:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493741568 00000100:00100000:0.0:1789760745.509674:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493741568 00000100:00100000:0.0:1789760745.509728:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755772391286 time 755812382035 state 0x0 ref 1 00000100:00100000:0.0:1789760745.509743:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755812642451 time 755812382035 state 0x1 ref 1 00000100:00100000:0.0:1789760745.509776:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755812642451 time 755850811870 state 0x0 ref 1 00000100:00100000:0.0:1789760745.509787:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493741568:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760745.509834:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24285184, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:0.0:1789760745.509843:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05849 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.509859:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760745.509864:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.509870:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05849 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.509885:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05849 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760745.543233:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24285184 -> 24289280 00010000:00010000:2.0:1789760745.543252:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05849 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.543266:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05849 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.543276:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05849 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.543284:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760745.543289:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05849 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.543299:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05849 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.543318:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05849 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.543327:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05849 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760745.543422:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493741568:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33631us (34136us total) trans 4294973248 rc 0/0 00000100:00100000:2.0:1789760745.543432:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5959 00000100:00100000:2.0:1789760745.543442:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755812642451 time 755850811870 state 0x2 ref 0 00000100:00100000:2.0:1789760745.546224:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000001:00080000:0.0:1789760745.546599:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973248 is committed 00000100:00100000:3.0:1789760745.546677:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493741696 00000100:00100000:3.0:1789760745.546687:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493741696 00000100:00100000:3.0:1789760745.546737:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755812642451 time 755850811870 state 0x0 ref 1 00000100:00100000:3.0:1789760745.546753:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755851072286 time 755850811870 state 0x1 ref 1 00000100:00100000:3.0:1789760745.546784:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755851072286 time 755887819509 state 0x0 ref 1 00000100:00100000:3.0:1789760745.546795:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493741696:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760745.546841:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24289280, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:3.0:1789760745.546850:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05850 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.546867:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760745.546871:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.546877:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05850 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.546890:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05850 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760745.588288:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973249 is committed 00002000:00010000:0.0:1789760745.588433:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24289280 -> 24293376 00010000:00010000:0.0:1789760745.588460:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05850 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.588658:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05850 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.588668:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05850 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.588682:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760745.588687:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05850 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.588697:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05850 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.588716:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05850 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.588724:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05850 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760745.588852:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493741696:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42054us (42641us total) trans 4294973249 rc 0/0 00000100:00100000:0.0:1789760745.588862:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5960 00000100:00100000:0.0:1789760745.588872:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755851072286 time 755887819509 state 0x2 ref 0 00000100:00100000:3.0:1789760745.592888:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760745.593412:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493741824 00000100:00100000:1.0:1789760745.593422:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493741824 00000100:00100000:0.0:1789760745.593690:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755851072286 time 755887819509 state 0x0 ref 1 00000100:00100000:0.0:1789760745.593705:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755888079925 time 755887819509 state 0x1 ref 1 00000100:00100000:0.0:1789760745.593762:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755888079925 time 755934797660 state 0x0 ref 1 00000100:00100000:0.0:1789760745.593774:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493741824:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760745.593818:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24293376, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xc007 00010000:00010000:0.0:1789760745.593827:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05857 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.593843:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760745.593847:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.593853:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05857 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.593866:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05857 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760745.637680:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24293376 -> 24297472 00002000:00010000:0.0:1789760745.637688:0:13038:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760744 -> 1789760745 00002000:00010000:0.0:1789760745.637691:0:13038:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760744 -> 1789760745 00010000:00010000:0.0:1789760745.637704:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05857 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760745.637725:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973250 is committed 00010000:00010000:0.0:1789760745.637781:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05857 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.637788:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05857 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.637793:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760745.637797:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05857 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.637804:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05857 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.637818:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05857 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.637822:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05857 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760745.637900:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493741824:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44123us (45017us total) trans 4294973250 rc 0/0 00000100:00100000:0.0:1789760745.637918:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5961 00000100:00100000:0.0:1789760745.638329:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755888079925 time 755934797660 state 0x2 ref 0 00000100:00100000:3.0:1789760745.644613:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760745.645010:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493741952 00000100:00100000:0.0:1789760745.645022:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493741952 00000100:00100000:0.0:1789760745.645079:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755888079925 time 755934797660 state 0x0 ref 1 00000100:00100000:0.0:1789760745.645094:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755935058076 time 755934797660 state 0x1 ref 1 00000100:00100000:0.0:1789760745.645125:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755935058076 time 755986161379 state 0x0 ref 1 00000100:00100000:0.0:1789760745.645138:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493741952:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760745.645187:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24297472, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:0.0:1789760745.645195:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0585e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.645212:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760745.645217:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.645224:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0585e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.645239:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0585e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760745.688323:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24297472 -> 24301568 00010000:00010000:0.0:1789760745.688342:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0585e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.688356:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0585e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.688365:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0585e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.688372:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760745.688377:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0585e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.688386:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0585e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.688407:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0585e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.688416:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0585e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760745.688524:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493741952:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43383us (43917us total) trans 4294973251 rc 0/0 00000100:00100000:0.0:1789760745.688534:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5962 00000100:00100000:0.0:1789760745.688543:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755935058076 time 755986161379 state 0x2 ref 0 00000001:00080000:1.0:1789760745.689352:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973251 is committed 00000100:00100000:2.0:1789760745.695437:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760745.695544:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493742080 00000100:00100000:1.0:1789760745.695555:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493742080 00000100:00100000:1.0:1789760745.695614:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755935058076 time 755986161379 state 0x0 ref 1 00000100:00100000:1.0:1789760745.695630:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755986421795 time 755986161379 state 0x1 ref 1 00000100:00100000:1.0:1789760745.695710:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755986421795 time 756036744349 state 0x0 ref 1 00000100:00100000:1.0:1789760745.695723:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493742080:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760745.695773:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24301568, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:1.0:1789760745.695781:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05865 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.695798:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:1.0:1789760745.695804:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760745.695810:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05865 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.695823:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05865 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760745.747505:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973252 is committed 00002000:00010000:0.0:1789760745.748841:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24301568 -> 24305664 00010000:00010000:0.0:1789760745.748858:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05865 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.748870:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05865 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.748878:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05865 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.748895:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760745.748900:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05865 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.748909:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05865 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.749223:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05865 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.749233:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05865 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760745.750568:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493742080:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54837us (55130us total) trans 4294973252 rc 0/0 00000100:00100000:0.0:1789760745.750580:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5963 00000100:00100000:0.0:1789760745.750594:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755986421795 time 756036744349 state 0x2 ref 0 00000100:00100000:2.0:1789760745.757021:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760745.757129:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493742208 00000100:00100000:3.0:1789760745.757138:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493742208 00000100:00100000:3.0:1789760745.757187:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 755986421795 time 756036744349 state 0x0 ref 1 00000100:00100000:3.0:1789760745.757202:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756037004765 time 756036744349 state 0x1 ref 1 00000100:00100000:3.0:1789760745.757313:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756037004765 time 756098346682 state 0x0 ref 1 00000100:00100000:3.0:1789760745.757326:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493742208:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760745.757375:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24305664, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:3.0:1789760745.757384:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0586c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.757400:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760745.757405:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.757420:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0586c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.757434:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0586c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760745.808205:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24305664 -> 24309760 00010000:00010000:0.0:1789760745.808227:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0586c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.808239:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0586c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.808247:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0586c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.808255:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760745.808260:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0586c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.808275:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0586c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.808295:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0586c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.808304:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0586c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760745.809129:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493742208:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51072us (51385us total) trans 4294973253 rc 0/0 00000100:00100000:0.0:1789760745.809141:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5964 00000100:00100000:0.0:1789760745.809152:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756037004765 time 756098346682 state 0x2 ref 0 00000001:00080000:1.0:1789760745.809239:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973253 is committed 00000100:00100000:0.0:1789760745.815425:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760745.815755:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493742336 00000100:00100000:1.0:1789760745.815763:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493742336 00000100:00100000:1.0:1789760745.815815:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756037004765 time 756098346682 state 0x0 ref 1 00000100:00100000:1.0:1789760745.815829:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756098607098 time 756098346682 state 0x1 ref 1 00000100:00100000:1.0:1789760745.815856:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756098607098 time 756156891674 state 0x0 ref 1 00000100:00100000:1.0:1789760745.815866:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493742336:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760745.815921:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24309760, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:1.0:1789760745.815929:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05873 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.815944:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:1.0:1789760745.815948:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760745.815954:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05873 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.815989:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05873 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760745.857913:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24309760 -> 24313856 00010000:00010000:1.0:1789760745.857945:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05873 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.857991:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05873 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.858008:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05873 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.858017:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760745.858022:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05873 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.858033:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05873 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.858053:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05873 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.858063:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05873 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.858148:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493742336:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42277us (42724us total) trans 4294973254 rc 0/0 00000100:00100000:1.0:1789760745.858158:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5965 00000100:00100000:1.0:1789760745.858168:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756098607098 time 756156891674 state 0x2 ref 0 00000001:00080000:1.0:1789760745.858206:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973254 is committed 00000100:00100000:2.0:1789760745.862322:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760745.862515:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493742464 00000100:00100000:3.0:1789760745.862526:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493742464 00000100:00100000:3.0:1789760745.862662:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756098607098 time 756156891674 state 0x0 ref 1 00000100:00100000:3.0:1789760745.862679:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756157152090 time 756156891674 state 0x1 ref 1 00000100:00100000:3.0:1789760745.862718:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756157152090 time 756203753108 state 0x0 ref 1 00000100:00100000:3.0:1789760745.862731:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493742464:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760745.862786:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24313856, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:3.0:1789760745.862794:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0587a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.862810:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760745.862815:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.862821:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0587a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.862836:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0587a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760745.896003:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24313856 -> 24317952 00010000:00010000:3.0:1789760745.896023:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0587a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.896037:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0587a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.896047:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0587a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.896055:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760745.896060:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0587a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.896069:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0587a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.896089:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0587a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.896097:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0587a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.896183:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493742464:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33449us (33865us total) trans 4294973255 rc 0/0 00000100:00100000:3.0:1789760745.896196:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5966 00000100:00100000:3.0:1789760745.896207:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756157152090 time 756203753108 state 0x2 ref 0 00000001:00080000:0.0:1789760745.896291:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973255 is committed 00000100:00100000:3.0:1789760745.914368:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760745.914916:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493742592 00000100:00100000:0.0:1789760745.915254:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493742592 00000100:00100000:0.0:1789760745.915307:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756157152090 time 756203753108 state 0x0 ref 1 00000100:00100000:0.0:1789760745.915321:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756204013524 time 756203753108 state 0x1 ref 1 00000100:00100000:0.0:1789760745.915341:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756204013524 time 756256377670 state 0x0 ref 1 00000100:00100000:0.0:1789760745.915352:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493742592:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760745.915401:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24317952, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:0.0:1789760745.915409:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05881 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.915425:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760745.915429:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.915436:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05881 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.915463:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05881 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760745.958787:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973256 is committed 00002000:00010000:0.0:1789760745.958864:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24317952 -> 24322048 00010000:00010000:0.0:1789760745.958878:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05881 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.958892:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05881 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.958901:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05881 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.958909:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760745.958914:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05881 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.959044:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05881 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.959066:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05881 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.959075:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05881 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760745.959190:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493742592:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43834us (44826us total) trans 4294973256 rc 0/0 00000100:00100000:0.0:1789760745.959200:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5967 00000100:00100000:0.0:1789760745.959209:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756204013524 time 756256377670 state 0x2 ref 0 00000100:00100000:3.0:1789760745.962938:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760745.963175:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493742720 00000100:00100000:0.0:1789760745.963185:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493742720 00000100:00100000:0.0:1789760745.963232:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756204013524 time 756256377670 state 0x0 ref 1 00000100:00100000:0.0:1789760745.963247:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756256638086 time 756256377670 state 0x1 ref 1 00000100:00100000:0.0:1789760745.963278:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756256638086 time 756304313295 state 0x0 ref 1 00000100:00100000:0.0:1789760745.963289:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493742720:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760745.963335:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24322048, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:0.0:1789760745.963344:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05888 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.963359:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760745.963364:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.963370:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05888 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.963385:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05888 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760745.998761:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973257 is committed 00002000:00010000:0.0:1789760745.999017:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24322048 -> 24326144 00010000:00010000:0.0:1789760745.999032:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05888 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.999044:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05888 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.999052:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05888 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.999059:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760745.999063:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05888 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.999071:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05888 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.999088:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05888 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.999094:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05888 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760745.999240:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493742720:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35948us (36318us total) trans 4294973257 rc 0/0 00000100:00100000:0.0:1789760745.999251:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5968 00000100:00100000:0.0:1789760745.999260:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756256638086 time 756304313295 state 0x2 ref 0 00000100:00100000:3.0:1789760746.003335:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760746.003768:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493742848 00000100:00100000:0.0:1789760746.003777:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493742848 00000100:00100000:0.0:1789760746.003910:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756256638086 time 756304313295 state 0x0 ref 1 00000100:00100000:0.0:1789760746.004092:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756304573711 time 756304313295 state 0x1 ref 1 00000100:00100000:0.0:1789760746.004165:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756304573711 time 756345199542 state 0x0 ref 1 00000100:00100000:0.0:1789760746.004178:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493742848:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760746.004225:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24326144, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:0.0:1789760746.004233:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0588f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.004250:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760746.004254:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.004260:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0588f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.004274:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0588f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.042492:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760746.044650:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973258 is committed 00002000:00010000:0.0:1789760746.044776:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24326144 -> 24330240 00010000:00010000:0.0:1789760746.044792:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0588f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.044871:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0588f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.044884:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0588f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.044891:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760746.044895:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0588f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.044905:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0588f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.044924:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0588f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.044932:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0588f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.045153:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493742848:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40972us (41822us total) trans 4294973258 rc 0/0 00000100:00100000:0.0:1789760746.045168:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5969 00000100:00100000:0.0:1789760746.045179:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756304573711 time 756345199542 state 0x2 ref 0 00000100:00100000:1.0:1789760746.050221:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760746.050330:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493742976 00000100:00100000:3.0:1789760746.050404:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493742976 00000100:00100000:3.0:1789760746.050465:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756304573711 time 756345199542 state 0x0 ref 1 00000100:00100000:3.0:1789760746.050480:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756345459958 time 756345199542 state 0x1 ref 1 00000100:00100000:3.0:1789760746.050604:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756345459958 time 756391638849 state 0x0 ref 1 00000100:00100000:3.0:1789760746.050618:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3661500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493742976:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760746.050665:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24330240, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:3.0:1789760746.050675:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05896 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.050690:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760746.050696:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.050702:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05896 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.050722:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05896 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760746.081660:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973259 is committed 00002000:00010000:1.0:1789760746.082002:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24330240 -> 24334336 00010000:00010000:1.0:1789760746.082018:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05896 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.082031:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05896 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.082040:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05896 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.082048:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760746.082053:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05896 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.082064:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05896 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.082083:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05896 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.082091:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05896 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.082225:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3661500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493742976:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31603us (32008us total) trans 4294973259 rc 0/0 00000100:00100000:1.0:1789760746.082236:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5970 00000100:00100000:1.0:1789760746.082246:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756345459958 time 756391638849 state 0x2 ref 0 00000100:00100000:3.0:1789760746.086151:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760746.086258:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493743104 00000100:00100000:1.0:1789760746.086267:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493743104 00000100:00100000:1.0:1789760746.086319:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756345459958 time 756391638849 state 0x0 ref 1 00000100:00100000:1.0:1789760746.086334:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756391899265 time 756391638849 state 0x1 ref 1 00000100:00100000:1.0:1789760746.086356:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756391899265 time 756427391625 state 0x0 ref 1 00000100:00100000:1.0:1789760746.086367:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493743104:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760746.086411:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24334336, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:1.0:1789760746.086420:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0589d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.086435:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:1.0:1789760746.086440:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760746.086460:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0589d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.086475:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0589d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760746.131170:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973260 is committed 00002000:00010000:1.0:1789760746.131349:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24334336 -> 24338432 00010000:00010000:1.0:1789760746.131366:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0589d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.131379:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0589d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.131389:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0589d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.131395:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760746.131400:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0589d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.131408:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0589d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.131428:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0589d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.131436:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e0589d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.131719:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493743104:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45347us (45571us total) trans 4294973260 rc 0/0 00000100:00100000:1.0:1789760746.131729:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5971 00000100:00100000:1.0:1789760746.131739:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756391899265 time 756427391625 state 0x2 ref 0 00000100:00100000:3.0:1789760746.137488:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760746.137767:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493743232 00000100:00100000:0.0:1789760746.137777:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493743232 00000100:00100000:0.0:1789760746.137841:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756391899265 time 756427391625 state 0x0 ref 1 00000100:00100000:0.0:1789760746.137856:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756427652041 time 756427391625 state 0x1 ref 1 00000100:00100000:0.0:1789760746.137888:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756427652041 time 756478923178 state 0x0 ref 1 00000100:00100000:0.0:1789760746.137899:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493743232:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760746.137948:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24338432, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:0.0:1789760746.137956:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e058a4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.138008:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760746.138013:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.138020:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e058a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.138034:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e058a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.193935:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973261 is committed 00002000:00010000:1.0:1789760746.196774:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24338432 -> 24342528 00010000:00010000:1.0:1789760746.196794:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e058a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.196813:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e058a4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.196821:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e058a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.196828:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760746.196832:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e058a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.196841:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e058a4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.196860:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e058a4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.196868:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e058a4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.197127:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493743232:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59223us (59641us total) trans 4294973261 rc 0/0 00000100:00100000:1.0:1789760746.197138:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5972 00000100:00100000:1.0:1789760746.197147:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756427652041 time 756478923178 state 0x2 ref 0 00000100:00100000:0.0:1789760746.200590:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760746.200744:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493743360 00000100:00100000:3.0:1789760746.200753:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493743360 00000100:00100000:3.0:1789760746.200808:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756427652041 time 756478923178 state 0x0 ref 1 00000100:00100000:3.0:1789760746.200822:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756479183594 time 756478923178 state 0x1 ref 1 00000100:00100000:3.0:1789760746.200851:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756479183594 time 756541887357 state 0x0 ref 1 00000100:00100000:3.0:1789760746.200864:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493743360:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760746.200916:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24342528, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:3.0:1789760746.200935:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e058ab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.200951:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760746.200956:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.200994:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e058ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.201009:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e058ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760746.233806:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24342528 -> 24346624 00010000:00010000:3.0:1789760746.233826:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e058ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.233837:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e058ab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.233846:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e058ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.233854:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760746.233859:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e058ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.233868:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e058ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.233886:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e058ab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.233895:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e058ab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.234029:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493743360:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33163us (33445us total) trans 4294973262 rc 0/0 00000100:00100000:3.0:1789760746.234043:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5973 00000100:00100000:3.0:1789760746.234052:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756479183594 time 756541887357 state 0x2 ref 0 00000001:00080000:2.0:1789760746.234729:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973262 is committed 00000100:00100000:0.0:1789760746.238297:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760746.238808:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493743488 00000100:00100000:1.0:1789760746.238819:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493743488 00000100:00100000:1.0:1789760746.238874:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756479183594 time 756541887357 state 0x0 ref 1 00000100:00100000:1.0:1789760746.238890:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756542147773 time 756541887357 state 0x1 ref 1 00000100:00100000:1.0:1789760746.241672:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756542147773 time 756582690359 state 0x0 ref 1 00000100:00100000:1.0:1789760746.241708:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493743488:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760746.241768:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24346624, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:1.0:1789760746.241777:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e058b2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.241792:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:1.0:1789760746.241797:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760746.241803:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e058b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.241818:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e058b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.282299:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973263 is committed 00002000:00010000:1.0:1789760746.282522:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24346624 -> 24350720 00010000:00010000:1.0:1789760746.282536:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e058b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.282549:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e058b2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.282564:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e058b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.282571:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760746.282576:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e058b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.282584:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e058b2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.282602:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e058b2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.282610:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e058b2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.282707:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493743488:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41011us (44413us total) trans 4294973263 rc 0/0 00000100:00100000:1.0:1789760746.282718:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5974 00000100:00100000:1.0:1789760746.282728:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756542147773 time 756582690359 state 0x2 ref 0 00000100:00100000:3.0:1789760746.289166:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760746.289263:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493743616 00000100:00100000:1.0:1789760746.289273:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493743616 00000100:00100000:1.0:1789760746.289329:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756542147773 time 756582690359 state 0x0 ref 1 00000100:00100000:1.0:1789760746.289344:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756582950775 time 756582690359 state 0x1 ref 1 00000100:00100000:1.0:1789760746.289378:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756582950775 time 756630413441 state 0x0 ref 1 00000100:00100000:1.0:1789760746.289391:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7aa7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493743616:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760746.289460:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24350720, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:1.0:1789760746.289469:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e058b9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.289485:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:1.0:1789760746.289490:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760746.289496:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e058b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.289509:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e058b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.337028:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973264 is committed 00002000:00010000:0.0:1789760746.337167:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24350720 -> 24354816 00010000:00010000:0.0:1789760746.337183:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e058b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.337196:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e058b9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.337205:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e058b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.337211:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760746.337216:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e058b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.337224:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e058b9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.337243:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e058b9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.337251:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e058b9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.337352:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7aa7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493743616:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47957us (48190us total) trans 4294973264 rc 0/0 00000100:00100000:0.0:1789760746.337363:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5975 00000100:00100000:0.0:1789760746.337372:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756582950775 time 756630413441 state 0x2 ref 0 00000100:00100000:2.0:1789760746.342215:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760746.342312:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493743744 00000100:00100000:1.0:1789760746.342322:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493743744 00000100:00100000:1.0:1789760746.342376:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756582950775 time 756630413441 state 0x0 ref 1 00000100:00100000:1.0:1789760746.342391:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756630673857 time 756630413441 state 0x1 ref 1 00000100:00100000:1.0:1789760746.342421:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756630673857 time 756683457152 state 0x0 ref 1 00000100:00100000:1.0:1789760746.342597:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493743744:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760746.342648:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24354816, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:1.0:1789760746.342658:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e058c0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.342673:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:1.0:1789760746.342677:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760746.342684:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e058c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.342698:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e058c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760746.374304:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24354816 -> 24358912 00010000:00010000:3.0:1789760746.374325:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e058c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.374337:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e058c0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.374345:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e058c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.374352:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760746.374356:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e058c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.374365:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e058c0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.374384:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e058c0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.374392:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e058c0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.374503:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493743744:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32060us (32285us total) trans 4294973265 rc 0/0 00000100:00100000:3.0:1789760746.374514:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5976 00000100:00100000:3.0:1789760746.374525:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756630673857 time 756683457152 state 0x2 ref 0 00000001:00080000:2.0:1789760746.374742:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973265 is committed 00000100:00100000:3.0:1789760746.379067:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760746.379215:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493743872 00000100:00100000:2.0:1789760746.379225:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493743872 00000100:00100000:2.0:1789760746.379282:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756630673857 time 756683457152 state 0x0 ref 1 00000100:00100000:2.0:1789760746.379297:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756683717568 time 756683457152 state 0x1 ref 1 00000100:00100000:2.0:1789760746.379327:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756683717568 time 756720362516 state 0x0 ref 1 00000100:00100000:2.0:1789760746.379345:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493743872:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760746.379393:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24358912, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:2.0:1789760746.379402:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e058c7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.379419:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760746.379424:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760746.379430:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e058c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.379452:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e058c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.424203:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973266 is committed 00002000:00010000:1.0:1789760746.424351:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24358912 -> 24363008 00010000:00010000:1.0:1789760746.424367:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e058c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.424450:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e058c7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.424461:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e058c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.424468:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760746.424473:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e058c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.424482:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e058c7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.424506:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e058c7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.424515:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e058c7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.424604:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493743872:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45257us (45544us total) trans 4294973266 rc 0/0 00000100:00100000:1.0:1789760746.424615:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5977 00000100:00100000:1.0:1789760746.424624:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756683717568 time 756720362516 state 0x2 ref 0 00000100:00100000:1.0:1789760746.428481:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760746.428651:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493744000 00000100:00100000:3.0:1789760746.428660:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493744000 00000100:00100000:3.0:1789760746.428711:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756683717568 time 756720362516 state 0x0 ref 1 00000100:00100000:3.0:1789760746.428725:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756720622932 time 756720362516 state 0x1 ref 1 00000100:00100000:3.0:1789760746.428760:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756720622932 time 756769795715 state 0x0 ref 1 00000100:00100000:3.0:1789760746.428772:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9621018a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493744000:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760746.428827:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24363008, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:3.0:1789760746.428836:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e058ce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.428850:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760746.428854:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.428861:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e058ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.428874:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e058ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760746.468243:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24363008 -> 24367104 00010000:00010000:0.0:1789760746.468263:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e058ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.468273:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e058ce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.468278:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e058ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.468283:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760746.468286:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e058ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.468292:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e058ce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.468306:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e058ce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.468314:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e058ce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.468394:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9621018a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493744000:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39619us (39917us total) trans 4294973267 rc 0/0 00000100:00100000:0.0:1789760746.468403:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5978 00000100:00100000:0.0:1789760746.468412:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756720622932 time 756769795715 state 0x2 ref 0 00000001:00080000:2.0:1789760746.468506:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973267 is committed 00000100:00100000:2.0:1789760746.472400:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760746.472661:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493744128 00000100:00100000:0.0:1789760746.472679:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493744128 00000100:00100000:0.0:1789760746.472734:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756720622932 time 756769795715 state 0x0 ref 1 00000100:00100000:0.0:1789760746.472749:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756770056131 time 756769795715 state 0x1 ref 1 00000100:00100000:0.0:1789760746.472781:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756770056131 time 756813817030 state 0x0 ref 1 00000100:00100000:0.0:1789760746.472798:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493744128:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760746.472842:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24367104, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:0.0:1789760746.472849:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e058d5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.472861:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760746.472864:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.472868:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e058d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.472882:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e058d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.522773:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973268 is committed 00002000:00010000:0.0:1789760746.522870:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24367104 -> 24371200 00010000:00010000:0.0:1789760746.522885:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e058d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.522898:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e058d5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.522906:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e058d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.522913:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760746.522918:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e058d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.523012:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e058d5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.523033:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e058d5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.523040:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e058d5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.523137:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493744128:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50342us (50742us total) trans 4294973268 rc 0/0 00000100:00100000:0.0:1789760746.523148:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5979 00000100:00100000:0.0:1789760746.523157:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756770056131 time 756813817030 state 0x2 ref 0 00000100:00100000:1.0:1789760746.526215:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760746.526311:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493744256 00000100:00100000:3.0:1789760746.526320:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493744256 00000100:00100000:3.0:1789760746.526375:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756770056131 time 756813817030 state 0x0 ref 1 00000100:00100000:3.0:1789760746.526393:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756814077446 time 756813817030 state 0x1 ref 1 00000100:00100000:3.0:1789760746.526470:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756814077446 time 756867504103 state 0x0 ref 1 00000100:00100000:3.0:1789760746.526484:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9621018a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493744256:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760746.527213:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24371200, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:3.0:1789760746.527223:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e058dc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.527240:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760746.527245:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.527251:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e058dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.527265:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e058dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760746.584391:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973269 is committed 00002000:00010000:1.0:1789760746.584704:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24371200 -> 24375296 00010000:00010000:1.0:1789760746.584719:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e058dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.584732:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e058dc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.584742:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e058dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.584750:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760746.584754:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e058dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.584765:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e058dc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.584782:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e058dc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.584790:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e058dc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.584878:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9621018a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493744256:12345-192.168.203.60@tcp:4:dd.0 Request processed in 58393us (58667us total) trans 4294973269 rc 0/0 00000100:00100000:1.0:1789760746.584888:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5980 00000100:00100000:1.0:1789760746.584898:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756814077446 time 756867504103 state 0x2 ref 0 00000100:00100000:1.0:1789760746.587742:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760746.587834:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493744384 00000100:00100000:2.0:1789760746.587845:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493744384 00000100:00100000:2.0:1789760746.587898:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756814077446 time 756867504103 state 0x0 ref 1 00000100:00100000:2.0:1789760746.587913:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756867764519 time 756867504103 state 0x1 ref 1 00000100:00100000:2.0:1789760746.587944:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756867764519 time 756928979613 state 0x0 ref 1 00000100:00100000:2.0:1789760746.587957:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8794700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493744384:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760746.588044:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24375296, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xc007 00010000:00010000:2.0:1789760746.588053:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e058e3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.588070:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760746.588075:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760746.588081:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e058e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.588096:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e058e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.623509:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973270 is committed 00002000:00010000:1.0:1789760746.623669:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24375296 -> 24379392 00002000:00010000:1.0:1789760746.623674:0:13038:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760745 -> 1789760746 00002000:00010000:1.0:1789760746.623677:0:13038:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760745 -> 1789760746 00010000:00010000:1.0:1789760746.623690:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e058e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.623704:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e058e3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.623778:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e058e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.623788:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760746.623794:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e058e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.623804:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e058e3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.623824:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e058e3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.623832:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e058e3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.623957:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8794700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493744384:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35996us (36218us total) trans 4294973270 rc 0/0 00000100:00100000:1.0:1789760746.624007:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5981 00000100:00100000:1.0:1789760746.624018:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756867764519 time 756928979613 state 0x2 ref 0 00000100:00100000:2.0:1789760746.631574:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760746.631679:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493744512 00000100:00100000:0.0:1789760746.631689:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493744512 00000100:00100000:0.0:1789760746.631756:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756867764519 time 756928979613 state 0x0 ref 1 00000100:00100000:0.0:1789760746.631773:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756929240029 time 756928979613 state 0x1 ref 1 00000100:00100000:0.0:1789760746.631807:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756929240029 time 756972842107 state 0x0 ref 1 00000100:00100000:0.0:1789760746.631820:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493744512:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760746.631872:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24379392, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:0.0:1789760746.631882:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e058ea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.631899:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760746.631904:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.631911:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e058ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.631926:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e058ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760746.670843:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24379392 -> 24383488 00010000:00010000:3.0:1789760746.670863:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e058ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.670881:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e058ea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.670889:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e058ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.670896:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760746.670901:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e058ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.670910:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e058ea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.671054:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e058ea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.671064:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e058ea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.671157:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493744512:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39334us (39588us total) trans 4294973271 rc 0/0 00000100:00100000:3.0:1789760746.671168:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5982 00000100:00100000:3.0:1789760746.671178:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756929240029 time 756972842107 state 0x2 ref 0 00000001:00080000:2.0:1789760746.672334:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973271 is committed 00000100:00100000:3.0:1789760746.675010:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760746.675122:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493744640 00000100:00100000:0.0:1789760746.675131:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493744640 00000100:00100000:0.0:1789760746.675183:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756929240029 time 756972842107 state 0x0 ref 1 00000100:00100000:0.0:1789760746.675198:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756973102523 time 756972842107 state 0x1 ref 1 00000100:00100000:0.0:1789760746.675228:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756973102523 time 757016263540 state 0x0 ref 1 00000100:00100000:0.0:1789760746.675241:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493744640:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760746.675287:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24383488, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:0.0:1789760746.675296:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e058f1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.675310:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760746.675314:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.675320:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e058f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.675335:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e058f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.701600:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973272 is committed 00002000:00010000:0.0:1789760746.701777:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24383488 -> 24387584 00010000:00010000:0.0:1789760746.701792:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e058f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.701810:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e058f1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.701819:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e058f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.701826:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760746.701831:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e058f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.701841:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e058f1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.701860:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e058f1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.701868:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e058f1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.701993:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493744640:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26715us (26953us total) trans 4294973272 rc 0/0 00000100:00100000:0.0:1789760746.702005:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5983 00000100:00100000:0.0:1789760746.702014:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756973102523 time 757016263540 state 0x2 ref 0 00000100:00100000:0.0:1789760746.705889:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760746.706076:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493744768 00000100:00100000:1.0:1789760746.706087:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493744768 00000100:00100000:1.0:1789760746.706143:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 756973102523 time 757016263540 state 0x0 ref 1 00000100:00100000:1.0:1789760746.706158:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757016523956 time 757016263540 state 0x1 ref 1 00000100:00100000:1.0:1789760746.706184:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757016523956 time 757047219910 state 0x0 ref 1 00000100:00100000:1.0:1789760746.706200:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493744768:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760746.706252:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24387584, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:1.0:1789760746.706261:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e058f8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.706276:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760746.706281:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760746.706288:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e058f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.706302:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e058f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760746.735340:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24387584 -> 24391680 00010000:00010000:1.0:1789760746.735359:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e058f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.735372:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e058f8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.735381:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e058f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.735388:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760746.735392:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e058f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.735402:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e058f8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.735420:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e058f8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.735426:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e058f8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.735581:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493744768:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29378us (29696us total) trans 4294973273 rc 0/0 00000100:00100000:1.0:1789760746.735591:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5984 00000100:00100000:1.0:1789760746.735600:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757016523956 time 757047219910 state 0x2 ref 0 00000001:00080000:2.0:1789760746.738170:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973273 is committed 00000100:00100000:2.0:1789760746.739152:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760746.739223:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493744896 00000100:00100000:1.0:1789760746.739233:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493744896 00000100:00100000:1.0:1789760746.739285:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757016523956 time 757047219910 state 0x0 ref 1 00000100:00100000:1.0:1789760746.739299:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757047480326 time 757047219910 state 0x1 ref 1 00000100:00100000:1.0:1789760746.739328:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757047480326 time 757080364224 state 0x0 ref 1 00000100:00100000:1.0:1789760746.739341:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493744896:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760746.739386:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24391680, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:1.0:1789760746.739394:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e058ff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.739409:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760746.739414:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760746.739420:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e058ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.739433:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e058ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760746.769930:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973274 is committed 00002000:00010000:3.0:1789760746.770044:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24391680 -> 24395776 00010000:00010000:3.0:1789760746.770059:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e058ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.770072:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e058ff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.770082:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e058ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.770090:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760746.770094:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e058ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.770103:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e058ff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.770122:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e058ff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.770129:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e058ff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.770222:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493744896:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30879us (31072us total) trans 4294973274 rc 0/0 00000100:00100000:3.0:1789760746.770232:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5985 00000100:00100000:3.0:1789760746.770242:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757047480326 time 757080364224 state 0x2 ref 0 00000100:00100000:3.0:1789760746.776383:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760746.776668:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493745024 00000100:00100000:0.0:1789760746.776679:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493745024 00000100:00100000:0.0:1789760746.776734:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757047480326 time 757080364224 state 0x0 ref 1 00000100:00100000:0.0:1789760746.776750:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757080624640 time 757080364224 state 0x1 ref 1 00000100:00100000:0.0:1789760746.776782:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757080624640 time 757117817924 state 0x0 ref 1 00000100:00100000:0.0:1789760746.776794:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493745024:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760746.776841:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24395776, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:0.0:1789760746.776851:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05906 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.776866:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760746.776870:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.776876:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05906 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.776890:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05906 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.812827:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973275 is committed 00002000:00010000:1.0:1789760746.812912:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24395776 -> 24399872 00010000:00010000:1.0:1789760746.813009:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05906 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.813022:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05906 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.813031:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05906 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.813038:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760746.813043:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05906 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.813051:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05906 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.813074:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05906 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.813081:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05906 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.813161:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493745024:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36364us (36785us total) trans 4294973275 rc 0/0 00000100:00100000:1.0:1789760746.813170:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5986 00000100:00100000:1.0:1789760746.813179:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757080624640 time 757117817924 state 0x2 ref 0 00000100:00100000:1.0:1789760746.816407:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760746.816517:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493745152 00000100:00100000:2.0:1789760746.816527:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493745152 00000100:00100000:2.0:1789760746.816580:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757080624640 time 757117817924 state 0x0 ref 1 00000100:00100000:2.0:1789760746.816594:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757118078340 time 757117817924 state 0x1 ref 1 00000100:00100000:2.0:1789760746.816624:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757118078340 time 757157660221 state 0x0 ref 1 00000100:00100000:2.0:1789760746.816638:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8796a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493745152:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760746.816685:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24399872, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:2.0:1789760746.816694:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0590d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.816709:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760746.816714:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760746.816720:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0590d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.816734:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0590d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760746.850420:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973276 is committed 00002000:00010000:2.0:1789760746.850705:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24399872 -> 24403968 00010000:00010000:2.0:1789760746.850720:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0590d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.850732:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0590d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.850741:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0590d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.850748:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760746.850753:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0590d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.850763:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0590d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.850782:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0590d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.850790:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0590d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760746.850934:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8796a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493745152:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34294us (34533us total) trans 4294973276 rc 0/0 00000100:00100000:2.0:1789760746.850946:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5987 00000100:00100000:2.0:1789760746.850956:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757118078340 time 757157660221 state 0x2 ref 0 00000100:00100000:2.0:1789760746.855327:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760746.855410:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493745280 00000100:00100000:1.0:1789760746.855420:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493745280 00000100:00100000:1.0:1789760746.855486:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757118078340 time 757157660221 state 0x0 ref 1 00000100:00100000:1.0:1789760746.855512:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757157920637 time 757157660221 state 0x1 ref 1 00000100:00100000:1.0:1789760746.855647:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757157920637 time 757196681852 state 0x0 ref 1 00000100:00100000:1.0:1789760746.855663:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493745280:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760746.855713:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24403968, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:1.0:1789760746.855723:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05914 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.855745:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760746.855750:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760746.855756:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05914 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.855770:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05914 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760746.888254:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24403968 -> 24408064 00010000:00010000:1.0:1789760746.888273:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05914 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.888287:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05914 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.888296:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05914 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.888304:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760746.888308:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05914 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.888314:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05914 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.888329:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05914 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.888335:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05914 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.888404:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493745280:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32741us (33082us total) trans 4294973277 rc 0/0 00000100:00100000:1.0:1789760746.888411:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5988 00000100:00100000:1.0:1789760746.888419:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757157920637 time 757196681852 state 0x2 ref 0 00000001:00080000:3.0:1789760746.888514:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973277 is committed 00000100:00100000:3.0:1789760746.893855:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760746.893947:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493745408 00000100:00100000:1.0:1789760746.893956:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493745408 00000100:00100000:1.0:1789760746.894042:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757157920637 time 757196681852 state 0x0 ref 1 00000100:00100000:1.0:1789760746.894056:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757196942268 time 757196681852 state 0x1 ref 1 00000100:00100000:1.0:1789760746.894238:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757196942268 time 757235272349 state 0x0 ref 1 00000100:00100000:1.0:1789760746.894252:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493745408:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760746.894299:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24408064, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:1.0:1789760746.894307:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0591b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.894322:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760746.894327:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760746.894337:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0591b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.894356:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0591b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760746.925470:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973278 is committed 00002000:00010000:3.0:1789760746.925605:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24408064 -> 24412160 00010000:00010000:3.0:1789760746.925620:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0591b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.925633:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0591b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.925642:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0591b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.925649:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760746.925653:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0591b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.925662:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0591b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.925680:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0591b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.925688:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0591b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.925768:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493745408:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31515us (31918us total) trans 4294973278 rc 0/0 00000100:00100000:3.0:1789760746.925775:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5989 00000100:00100000:3.0:1789760746.925784:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757196942268 time 757235272349 state 0x2 ref 0 00000100:00100000:1.0:1789760746.934087:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760746.934187:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493745536 00000100:00100000:2.0:1789760746.934196:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493745536 00000100:00100000:2.0:1789760746.934252:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757196942268 time 757235272349 state 0x0 ref 1 00000100:00100000:2.0:1789760746.934268:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757235532765 time 757235272349 state 0x1 ref 1 00000100:00100000:2.0:1789760746.934298:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757235532765 time 757275333952 state 0x0 ref 1 00000100:00100000:2.0:1789760746.934310:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493745536:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760746.934358:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24412160, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:2.0:1789760746.934367:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05922 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.934389:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760746.934393:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760746.934399:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05922 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.934413:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05922 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760746.962401:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24412160 -> 24416256 00010000:00010000:1.0:1789760746.962421:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05922 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.962434:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05922 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.962453:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05922 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.962460:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760746.962465:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05922 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.962474:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05922 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.962501:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05922 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.962509:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05922 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.962598:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493745536:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28286us (28518us total) trans 4294973279 rc 0/0 00000100:00100000:1.0:1789760746.962609:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5990 00000100:00100000:1.0:1789760746.962619:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757235532765 time 757275333952 state 0x2 ref 0 00000001:00080000:1.0:1789760746.962658:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973279 is committed 00000100:00100000:2.0:1789760746.967862:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760746.967992:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493745664 00000100:00100000:0.0:1789760746.968002:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493745664 00000100:00100000:0.0:1789760746.968067:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757235532765 time 757275333952 state 0x0 ref 1 00000100:00100000:0.0:1789760746.968083:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757275594368 time 757275333952 state 0x1 ref 1 00000100:00100000:0.0:1789760746.968113:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757275594368 time 757309148646 state 0x0 ref 1 00000100:00100000:0.0:1789760746.968125:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493745664:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760746.968172:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24416256, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:0.0:1789760746.968182:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05929 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.968197:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760746.968202:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.968207:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05929 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.968221:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05929 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760747.025421:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973280 is committed 00002000:00010000:0.0:1789760747.025537:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24416256 -> 24420352 00010000:00010000:0.0:1789760747.025552:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05929 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.025567:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05929 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.025577:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05929 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.025585:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760747.025590:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05929 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.025600:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05929 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.025620:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05929 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.025627:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05929 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.027104:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493745664:12345-192.168.203.60@tcp:4:dd.0 Request processed in 58971us (59242us total) trans 4294973280 rc 0/0 00000100:00100000:0.0:1789760747.027119:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5991 00000100:00100000:0.0:1789760747.027139:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757275594368 time 757309148646 state 0x2 ref 0 00000100:00100000:1.0:1789760747.030677:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760747.030754:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493745792 00000100:00100000:2.0:1789760747.030763:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493745792 00000100:00100000:2.0:1789760747.030854:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757275594368 time 757309148646 state 0x0 ref 1 00000100:00100000:2.0:1789760747.030869:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757309409062 time 757309148646 state 0x1 ref 1 00000100:00100000:2.0:1789760747.030898:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757309409062 time 757371933873 state 0x0 ref 1 00000100:00100000:2.0:1789760747.030911:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493745792:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760747.030987:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24420352, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:2.0:1789760747.030996:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05930 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.031012:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760747.031016:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.031022:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05930 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.031035:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05930 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760747.032343:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789760747.066085:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760747.066515:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24420352 -> 24424448 00010000:00010000:1.0:1789760747.066639:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05930 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.066652:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05930 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.066660:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05930 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.066667:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760747.066671:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05930 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.066680:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05930 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.066705:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05930 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.066713:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05930 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760747.066834:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493745792:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35921us (36159us total) trans 4294973281 rc 0/0 00000100:00100000:1.0:1789760747.066845:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5992 00000100:00100000:1.0:1789760747.066855:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757309409062 time 757371933873 state 0x2 ref 0 00000001:00080000:3.0:1789760747.066912:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973281 is committed 00000100:00100000:1.0:1789760747.071705:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760747.071777:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493745920 00000100:00100000:3.0:1789760747.071787:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493745920 00000100:00100000:3.0:1789760747.071840:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757309409062 time 757371933873 state 0x0 ref 1 00000100:00100000:3.0:1789760747.071856:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757372194289 time 757371933873 state 0x1 ref 1 00000100:00100000:3.0:1789760747.071927:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757372194289 time 757412953970 state 0x0 ref 1 00000100:00100000:3.0:1789760747.071940:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493745920:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760747.072015:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24424448, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:3.0:1789760747.072025:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05937 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.072040:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760747.072044:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.072050:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05937 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.072064:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05937 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760747.107604:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24424448 -> 24428544 00010000:00010000:0.0:1789760747.107624:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05937 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.107637:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05937 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.107646:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05937 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.107653:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760747.107658:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05937 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.107666:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05937 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.107684:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05937 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.107690:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05937 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.107778:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493745920:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35834us (36077us total) trans 4294973282 rc 0/0 00000100:00100000:0.0:1789760747.107789:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5993 00000100:00100000:0.0:1789760747.107799:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757372194289 time 757412953970 state 0x2 ref 0 00000001:00080000:2.0:1789760747.107876:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973282 is committed 00000100:00100000:2.0:1789760747.111279:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760747.111352:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493746048 00000100:00100000:3.0:1789760747.111360:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493746048 00000100:00100000:3.0:1789760747.111405:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757372194289 time 757412953970 state 0x0 ref 1 00000100:00100000:3.0:1789760747.111418:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757413214386 time 757412953970 state 0x1 ref 1 00000100:00100000:3.0:1789760747.111453:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757413214386 time 757452489087 state 0x0 ref 1 00000100:00100000:3.0:1789760747.111464:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493746048:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760747.111516:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24428544, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:3.0:1789760747.111524:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0593e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.111539:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760747.111543:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.111549:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0593e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.111560:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0593e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760747.142429:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973283 is committed 00002000:00010000:0.0:1789760747.142542:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24428544 -> 24432640 00010000:00010000:0.0:1789760747.142557:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0593e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.142569:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0593e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.142577:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0593e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.142590:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760747.142594:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0593e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.142602:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0593e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.142620:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0593e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.142627:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0593e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.142724:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493746048:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31255us (31447us total) trans 4294973283 rc 0/0 00000100:00100000:0.0:1789760747.142734:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5994 00000100:00100000:0.0:1789760747.142744:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757413214386 time 757452489087 state 0x2 ref 0 00000100:00100000:3.0:1789760747.155025:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760747.155124:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493746176 00000100:00100000:0.0:1789760747.155133:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493746176 00000100:00100000:0.0:1789760747.155186:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757413214386 time 757452489087 state 0x0 ref 1 00000100:00100000:0.0:1789760747.155200:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757452749503 time 757452489087 state 0x1 ref 1 00000100:00100000:0.0:1789760747.155233:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757452749503 time 757496267837 state 0x0 ref 1 00000100:00100000:0.0:1789760747.155249:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493746176:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760747.155301:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24432640, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:0.0:1789760747.155311:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05945 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.155328:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760747.155333:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.155339:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05945 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.155352:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05945 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760747.194647:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24432640 -> 24436736 00010000:00010000:2.0:1789760747.194669:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05945 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.194682:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05945 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.194691:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05945 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.194698:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760747.194703:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05945 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.194711:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05945 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.194729:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05945 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.194743:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05945 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.194847:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493746176:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39596us (39826us total) trans 4294973284 rc 0/0 00000100:00100000:2.0:1789760747.194858:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5995 00000100:00100000:2.0:1789760747.194868:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757452749503 time 757496267837 state 0x2 ref 0 00000001:00080000:2.0:1789760747.194916:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973284 is committed 00000100:00100000:2.0:1789760747.202596:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760747.203202:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493746304 00000100:00100000:1.0:1789760747.203211:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493746304 00000100:00100000:1.0:1789760747.203268:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757452749503 time 757496267837 state 0x0 ref 1 00000100:00100000:1.0:1789760747.203285:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757496528253 time 757496267837 state 0x1 ref 1 00000100:00100000:1.0:1789760747.203606:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757496528253 time 757544640332 state 0x0 ref 1 00000100:00100000:1.0:1789760747.203620:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493746304:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760747.203669:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24436736, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:1.0:1789760747.203678:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0594c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.203698:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760747.203702:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760747.203709:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0594c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.203721:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0594c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760747.246736:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973285 is committed 00002000:00010000:2.0:1789760747.246849:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24436736 -> 24440832 00010000:00010000:2.0:1789760747.246864:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0594c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.246877:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0594c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.246887:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0594c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.246895:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760747.246900:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0594c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.246909:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0594c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.247168:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0594c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.247176:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0594c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.247380:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493746304:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43757us (44880us total) trans 4294973285 rc 0/0 00000100:00100000:3.0:1789760747.247392:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5996 00000100:00100000:3.0:1789760747.247402:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757496528253 time 757544640332 state 0x2 ref 0 00000100:00100000:0.0:1789760747.251454:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760747.251575:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493746432 00000100:00100000:3.0:1789760747.251584:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493746432 00000100:00100000:3.0:1789760747.251639:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757496528253 time 757544640332 state 0x0 ref 1 00000100:00100000:3.0:1789760747.251653:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757544900748 time 757544640332 state 0x1 ref 1 00000100:00100000:3.0:1789760747.251683:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757544900748 time 757592718524 state 0x0 ref 1 00000100:00100000:3.0:1789760747.251695:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493746432:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760747.251740:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24440832, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:3.0:1789760747.251749:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05953 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.251763:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760747.251768:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.251774:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05953 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.251787:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05953 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760747.299479:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973286 is committed 00002000:00010000:2.0:1789760747.299820:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24440832 -> 24444928 00010000:00010000:2.0:1789760747.300147:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05953 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.300159:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05953 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.300168:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05953 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.300175:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760747.300180:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05953 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.300188:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05953 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.300207:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05953 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.300220:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05953 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.300396:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493746432:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48697us (48954us total) trans 4294973286 rc 0/0 00000100:00100000:3.0:1789760747.300407:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5997 00000100:00100000:3.0:1789760747.300416:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757544900748 time 757592718524 state 0x2 ref 0 00000100:00100000:1.0:1789760747.304374:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760747.304489:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493746560 00000100:00100000:0.0:1789760747.304506:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493746560 00000100:00100000:0.0:1789760747.304566:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757544900748 time 757592718524 state 0x0 ref 1 00000100:00100000:0.0:1789760747.304580:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757592978940 time 757592718524 state 0x1 ref 1 00000100:00100000:0.0:1789760747.304610:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757592978940 time 757645645192 state 0x0 ref 1 00000100:00100000:0.0:1789760747.304622:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493746560:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760747.304668:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24444928, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:0.0:1789760747.304678:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e0595a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.304692:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760747.304697:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.304703:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e0595a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.304716:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e0595a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760747.336171:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24444928 -> 24449024 00010000:00010000:3.0:1789760747.336191:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e0595a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.336205:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e0595a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.336214:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e0595a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.336221:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760747.336226:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e0595a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.336235:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e0595a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.336254:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e0595a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.336262:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e0595a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.336352:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493746560:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31726us (31983us total) trans 4294973287 rc 0/0 00000100:00100000:3.0:1789760747.336362:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5998 00000100:00100000:3.0:1789760747.336372:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757592978940 time 757645645192 state 0x2 ref 0 00000001:00080000:0.0:1789760747.336460:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973287 is committed 00000100:00100000:2.0:1789760747.341176:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760747.341691:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493746688 00000100:00100000:0.0:1789760747.341701:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493746688 00000100:00100000:0.0:1789760747.341759:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757592978940 time 757645645192 state 0x0 ref 1 00000100:00100000:0.0:1789760747.341777:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757645905608 time 757645645192 state 0x1 ref 1 00000100:00100000:0.0:1789760747.341808:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757645905608 time 757682843475 state 0x0 ref 1 00000100:00100000:0.0:1789760747.341821:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493746688:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760747.341870:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24449024, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:0.0:1789760747.341889:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05961 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.341905:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760747.341910:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.342152:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05961 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.342169:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05961 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760747.373073:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973288 is committed 00002000:00010000:2.0:1789760747.373242:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24449024 -> 24453120 00010000:00010000:2.0:1789760747.373258:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05961 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.373270:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05961 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.373278:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05961 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.373284:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760747.373289:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05961 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.373298:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05961 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.373314:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05961 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.373321:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05961 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.373410:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493746688:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31587us (32239us total) trans 4294973288 rc 0/0 00000100:00100000:2.0:1789760747.373420:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 5999 00000100:00100000:2.0:1789760747.373430:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757645905608 time 757682843475 state 0x2 ref 0 00000100:00100000:2.0:1789760747.376633:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760747.376847:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493746816 00000100:00100000:0.0:1789760747.376857:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493746816 00000100:00100000:0.0:1789760747.376907:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757645905608 time 757682843475 state 0x0 ref 1 00000100:00100000:0.0:1789760747.376935:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757683103891 time 757682843475 state 0x1 ref 1 00000100:00100000:0.0:1789760747.376989:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757683103891 time 757718024427 state 0x0 ref 1 00000100:00100000:0.0:1789760747.377001:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493746816:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760747.377046:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24453120, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:0.0:1789760747.377054:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05968 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.377068:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760747.377072:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.377078:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05968 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.377091:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05968 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760747.409621:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973289 is committed 00002000:00010000:3.0:1789760747.409727:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24453120 -> 24457216 00010000:00010000:3.0:1789760747.409743:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05968 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.409756:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05968 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.409764:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05968 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.409772:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760747.409777:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05968 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.409786:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05968 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.409805:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05968 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.409813:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05968 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.409901:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493746816:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32897us (33273us total) trans 4294973289 rc 0/0 00000100:00100000:3.0:1789760747.410023:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6000 00000100:00100000:3.0:1789760747.410033:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757683103891 time 757718024427 state 0x2 ref 0 00000100:00100000:3.0:1789760747.412897:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760747.413355:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493746944 00000100:00100000:0.0:1789760747.413365:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493746944 00000100:00100000:0.0:1789760747.413418:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757683103891 time 757718024427 state 0x0 ref 1 00000100:00100000:0.0:1789760747.413442:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757718284843 time 757718024427 state 0x1 ref 1 00000100:00100000:0.0:1789760747.413475:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757718284843 time 757754509119 state 0x0 ref 1 00000100:00100000:0.0:1789760747.413498:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493746944:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760747.413547:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24457216, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:0.0:1789760747.413557:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0596f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.413574:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760747.413578:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.413584:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0596f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.413599:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0596f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760747.443644:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973290 is committed 00002000:00010000:3.0:1789760747.443739:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24457216 -> 24461312 00010000:00010000:3.0:1789760747.443753:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0596f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.443767:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0596f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.443776:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0596f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.443784:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760747.443789:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0596f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.443799:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0596f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.443819:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0596f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.443826:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0596f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.443914:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493746944:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30423us (31023us total) trans 4294973290 rc 0/0 00000100:00100000:3.0:1789760747.443923:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6001 00000100:00100000:3.0:1789760747.443934:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757718284843 time 757754509119 state 0x2 ref 0 00000100:00100000:2.0:1789760747.449569:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760747.449839:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493747072 00000100:00100000:3.0:1789760747.449849:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493747072 00000100:00100000:3.0:1789760747.449904:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757718284843 time 757754509119 state 0x0 ref 1 00000100:00100000:3.0:1789760747.450102:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757754769535 time 757754509119 state 0x1 ref 1 00000100:00100000:3.0:1789760747.450127:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757754769535 time 757791163210 state 0x0 ref 1 00000100:00100000:3.0:1789760747.450141:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493747072:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760747.450191:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24461312, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:3.0:1789760747.450200:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05976 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.450215:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760747.450220:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.450226:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05976 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.450240:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05976 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760747.488223:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24461312 -> 24465408 00010000:00010000:3.0:1789760747.488245:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05976 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.488259:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05976 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.488268:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05976 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.488276:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760747.488282:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05976 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.488292:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05976 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.488312:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05976 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.488321:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05976 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.488415:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493747072:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38271us (38850us total) trans 4294973291 rc 0/0 00000100:00100000:3.0:1789760747.488429:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6002 00000100:00100000:3.0:1789760747.488447:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757754769535 time 757791163210 state 0x2 ref 0 00000001:00080000:1.0:1789760747.488912:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973291 is committed 00000100:00100000:0.0:1789760747.492230:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760747.492326:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493747200 00000100:00100000:2.0:1789760747.492336:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493747200 00000100:00100000:2.0:1789760747.492390:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757754769535 time 757791163210 state 0x0 ref 1 00000100:00100000:2.0:1789760747.492405:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757791423626 time 757791163210 state 0x1 ref 1 00000100:00100000:2.0:1789760747.492445:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757791423626 time 757833470896 state 0x0 ref 1 00000100:00100000:2.0:1789760747.492458:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493747200:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760747.492503:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24465408, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:2.0:1789760747.492512:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0597d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.492527:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760747.492531:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.492538:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0597d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.492557:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0597d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760747.525298:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973292 is committed 00002000:00010000:3.0:1789760747.525396:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24465408 -> 24469504 00010000:00010000:3.0:1789760747.525412:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0597d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.525425:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0597d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.525444:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0597d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.525453:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760747.525458:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0597d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.525466:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0597d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.525499:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0597d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.525508:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0597d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.525613:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493747200:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33153us (33388us total) trans 4294973292 rc 0/0 00000100:00100000:3.0:1789760747.525625:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6003 00000100:00100000:3.0:1789760747.525634:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757791423626 time 757833470896 state 0x2 ref 0 00000100:00100000:0.0:1789760747.529832:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760747.529877:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493747328 00000100:00100000:3.0:1789760747.529886:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493747328 00000100:00100000:3.0:1789760747.531630:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757791423626 time 757833470896 state 0x0 ref 1 00000100:00100000:3.0:1789760747.531647:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757833731312 time 757833470896 state 0x1 ref 1 00000100:00100000:3.0:1789760747.531751:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757833731312 time 757872786221 state 0x0 ref 1 00000100:00100000:3.0:1789760747.531769:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493747328:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760747.531819:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24469504, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:3.0:1789760747.531827:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05984 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.531842:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760747.531846:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.531852:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05984 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.531864:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05984 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760747.561617:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24469504 -> 24473600 00010000:00010000:2.0:1789760747.561640:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05984 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.561653:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05984 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.561663:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05984 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.561671:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760747.561676:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05984 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.561684:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05984 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.561704:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05984 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.561713:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05984 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.561809:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493747328:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30041us (31980us total) trans 4294973293 rc 0/0 00000100:00100000:2.0:1789760747.561821:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6004 00000100:00100000:2.0:1789760747.561830:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757833731312 time 757872786221 state 0x2 ref 0 00000001:00080000:1.0:1789760747.562128:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973293 is committed 00000100:00100000:2.0:1789760747.565183:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760747.565265:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493747456 00000100:00100000:3.0:1789760747.565275:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493747456 00000100:00100000:3.0:1789760747.565328:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757833731312 time 757872786221 state 0x0 ref 1 00000100:00100000:3.0:1789760747.565343:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757873046637 time 757872786221 state 0x1 ref 1 00000100:00100000:3.0:1789760747.565374:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757873046637 time 757906409445 state 0x0 ref 1 00000100:00100000:3.0:1789760747.565386:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493747456:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760747.565445:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24473600, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:3.0:1789760747.565455:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0598b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.565470:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760747.565474:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.565481:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0598b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.565506:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0598b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760747.593693:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973294 is committed 00002000:00010000:2.0:1789760747.593848:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24473600 -> 24477696 00010000:00010000:2.0:1789760747.593862:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0598b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.593876:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0598b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.593885:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0598b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.593893:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760747.593898:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0598b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.593908:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0598b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.593926:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0598b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.593939:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0598b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.594070:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493747456:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28677us (28889us total) trans 4294973294 rc 0/0 00000100:00100000:2.0:1789760747.594080:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6005 00000100:00100000:2.0:1789760747.594091:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757873046637 time 757906409445 state 0x2 ref 0 00000100:00100000:2.0:1789760747.604544:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760747.604705:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493747584 00000100:00100000:1.0:1789760747.604715:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493747584 00000100:00100000:1.0:1789760747.604770:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757873046637 time 757906409445 state 0x0 ref 1 00000100:00100000:1.0:1789760747.604785:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757906669861 time 757906409445 state 0x1 ref 1 00000100:00100000:1.0:1789760747.604815:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757906669861 time 757945850252 state 0x0 ref 1 00000100:00100000:1.0:1789760747.604828:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493747584:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760747.604876:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24477696, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xc007 00010000:00010000:1.0:1789760747.604885:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05992 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.604902:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760747.604906:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760747.604914:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05992 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.604927:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05992 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.624204:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760747.624243:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760747.624268:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760747.624301:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493747840 00000100:00100000:2.0:1789760747.624310:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493747840 00000100:00100000:2.0:1789760747.624365:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493747968 00000100:00100000:2.0:1789760747.624368:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493747968 00000100:00100000:2.0:1789760747.624395:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 487 00000100:00100000:2.0:1789760747.624405:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2370:x1876699493747840:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:2.0:1789760747.624482:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2370:x1876699493747840:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 79us (285us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760747.624607:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 487 00000100:00100000:2.0:1789760747.624625:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 488 00000100:00100000:2.0:1789760747.624633:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2370:x1876699493747968:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:2.0:1789760747.624661:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2370:x1876699493747968:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 29us (393us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760747.624736:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 488 00000100:00100000:2.0:1789760747.624756:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493747712 00000100:00100000:2.0:1789760747.624759:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493747712 00000100:00100000:2.0:1789760747.624787:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 401 00000100:00100000:2.0:1789760747.624793:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d4a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2370:x1876699493747712:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:2.0:1789760747.624820:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d4a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2370:x1876699493747712:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 28us (576us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760747.624826:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 401 00000100:00080000:2.0:1789760747.624855:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:3.0:1789760747.663337:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973295 is committed 00002000:00010000:0.0:1789760747.663449:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24477696 -> 24481792 00002000:00010000:0.0:1789760747.663455:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760746 -> 1789760747 00002000:00010000:0.0:1789760747.663458:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760746 -> 1789760747 00010000:00010000:0.0:1789760747.663470:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05992 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.663493:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05992 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.663503:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05992 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.663510:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760747.663514:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05992 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.663523:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05992 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.663545:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05992 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.663553:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05992 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.663649:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493747584:12345-192.168.203.60@tcp:4:dd.0 Request processed in 58818us (59111us total) trans 4294973295 rc 0/0 00000100:00100000:0.0:1789760747.663668:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6006 00000100:00100000:0.0:1789760747.663679:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757906669861 time 757945850252 state 0x2 ref 0 00000100:00100000:2.0:1789760747.667950:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760747.668092:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493748096 00000100:00100000:3.0:1789760747.668103:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493748096 00000100:00100000:3.0:1789760747.668160:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757906669861 time 757945850252 state 0x0 ref 1 00000100:00100000:3.0:1789760747.668180:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757946110668 time 757945850252 state 0x1 ref 1 00000100:00100000:3.0:1789760747.668214:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757946110668 time 758009250019 state 0x0 ref 1 00000100:00100000:3.0:1789760747.668227:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493748096:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760747.668276:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24481792, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc007 00010000:00010000:3.0:1789760747.668285:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05999 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.668302:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760747.668306:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.668313:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05999 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.668328:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05999 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760747.701081:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973296 is committed 00002000:00010000:3.0:1789760747.701187:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24481792 -> 24485888 00010000:00010000:3.0:1789760747.701202:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05999 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.701215:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05999 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.701225:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05999 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.701238:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760747.701242:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05999 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.701253:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05999 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.701272:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05999 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.701280:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05999 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.701371:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493748096:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33141us (33426us total) trans 4294973296 rc 0/0 00000100:00100000:3.0:1789760747.701382:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6007 00000100:00100000:3.0:1789760747.701391:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757946110668 time 758009250019 state 0x2 ref 0 00000100:00100000:3.0:1789760747.707228:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760747.707378:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493748224 00000100:00100000:2.0:1789760747.707389:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493748224 00000100:00100000:2.0:1789760747.707672:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 757946110668 time 758009250019 state 0x0 ref 1 00000100:00100000:2.0:1789760747.707688:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758009510435 time 758009250019 state 0x1 ref 1 00000100:00100000:2.0:1789760747.707771:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758009510435 time 758048805757 state 0x0 ref 1 00000100:00100000:2.0:1789760747.707786:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493748224:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760747.707835:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24485888, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc007 00010000:00010000:2.0:1789760747.707843:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e059a0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.707860:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760747.707864:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.707870:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e059a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.707885:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e059a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760747.738945:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973297 is committed 00002000:00010000:3.0:1789760747.739071:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24485888 -> 24489984 00010000:00010000:3.0:1789760747.739086:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e059a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.739098:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e059a0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.739121:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e059a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.739129:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760747.739134:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e059a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.739144:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e059a0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.739170:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e059a0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.739177:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e059a0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.739264:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493748224:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31474us (32038us total) trans 4294973297 rc 0/0 00000100:00100000:3.0:1789760747.739277:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6008 00000100:00100000:3.0:1789760747.739287:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758009510435 time 758048805757 state 0x2 ref 0 00000100:00100000:0.0:1789760747.742288:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760747.742449:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493748352 00000100:00100000:1.0:1789760747.742458:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493748352 00000100:00100000:1.0:1789760747.742510:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758009510435 time 758048805757 state 0x0 ref 1 00000100:00100000:1.0:1789760747.742524:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758049066173 time 758048805757 state 0x1 ref 1 00000100:00100000:1.0:1789760747.742552:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758049066173 time 758083588010 state 0x0 ref 1 00000100:00100000:1.0:1789760747.742564:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493748352:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760747.742948:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24489984, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc007 00010000:00010000:1.0:1789760747.743050:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e059a7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.743128:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760747.743134:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760747.743140:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e059a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.743196:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e059a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760747.780386:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973298 is committed 00002000:00010000:0.0:1789760747.780460:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24489984 -> 24494080 00010000:00010000:0.0:1789760747.780474:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e059a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.780547:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e059a7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.780558:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e059a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.780565:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760747.780570:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e059a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.780580:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e059a7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.780600:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e059a7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.780608:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e059a7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.780708:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493748352:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38140us (38423us total) trans 4294973298 rc 0/0 00000100:00100000:0.0:1789760747.780719:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6009 00000100:00100000:0.0:1789760747.780729:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758049066173 time 758083588010 state 0x2 ref 0 00000100:00100000:2.0:1789760747.784617:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760747.784710:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493748480 00000100:00100000:0.0:1789760747.784719:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493748480 00000100:00100000:0.0:1789760747.784773:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758049066173 time 758083588010 state 0x0 ref 1 00000100:00100000:0.0:1789760747.784788:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758083848426 time 758083588010 state 0x1 ref 1 00000100:00100000:0.0:1789760747.784852:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758083848426 time 758125887029 state 0x0 ref 1 00000100:00100000:0.0:1789760747.784864:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493748480:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760747.784911:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24494080, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc007 00010000:00010000:0.0:1789760747.784919:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e059ae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.784940:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760747.784945:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.784951:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e059ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.789071:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e059ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760747.829941:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973299 is committed 00002000:00010000:0.0:1789760747.830085:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24494080 -> 24498176 00010000:00010000:0.0:1789760747.830100:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e059ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.830114:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e059ae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.830123:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e059ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.830131:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760747.830136:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e059ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.830146:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e059ae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.830308:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e059ae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.830318:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e059ae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.830412:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493748480:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45544us (45800us total) trans 4294973299 rc 0/0 00000100:00100000:0.0:1789760747.830423:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6010 00000100:00100000:0.0:1789760747.830998:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758083848426 time 758125887029 state 0x2 ref 0 00000100:00100000:2.0:1789760747.839552:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760747.840176:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493748608 00000100:00100000:3.0:1789760747.840187:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493748608 00000100:00100000:3.0:1789760747.840253:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758083848426 time 758125887029 state 0x0 ref 1 00000100:00100000:3.0:1789760747.840270:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758126147445 time 758125887029 state 0x1 ref 1 00000100:00100000:3.0:1789760747.840304:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758126147445 time 758181339700 state 0x0 ref 1 00000100:00100000:3.0:1789760747.840319:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb04850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493748608:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760747.840369:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24498176, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc007 00010000:00010000:3.0:1789760747.840379:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e059b5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.840395:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760747.840400:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.840406:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e059b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.840421:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e059b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760747.882227:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973300 is committed 00002000:00010000:2.0:1789760747.882451:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24498176 -> 24502272 00010000:00010000:2.0:1789760747.882466:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e059b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.882480:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e059b5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.882670:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e059b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.882678:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760747.882682:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e059b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.882692:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e059b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.882713:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e059b5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.882727:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e059b5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.882865:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb04850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493748608:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42545us (43318us total) trans 4294973300 rc 0/0 00000100:00100000:2.0:1789760747.882875:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6011 00000100:00100000:2.0:1789760747.882885:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758126147445 time 758181339700 state 0x2 ref 0 00000100:00100000:2.0:1789760747.886805:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760747.887171:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493748736 00000100:00100000:3.0:1789760747.887181:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493748736 00000100:00100000:3.0:1789760747.887235:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758126147445 time 758181339700 state 0x0 ref 1 00000100:00100000:3.0:1789760747.887251:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758181600116 time 758181339700 state 0x1 ref 1 00000100:00100000:3.0:1789760747.887316:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758181600116 time 758228351397 state 0x0 ref 1 00000100:00100000:3.0:1789760747.887330:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493748736:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760747.887380:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24502272, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc007 00010000:00010000:3.0:1789760747.887390:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e059bc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.887406:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760747.887411:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.887416:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e059bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.887439:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e059bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760747.951909:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973301 is committed 00002000:00010000:2.0:1789760747.951927:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24502272 -> 24506368 00010000:00010000:2.0:1789760747.951942:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e059bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.951954:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e059bc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.951983:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e059bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.951999:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760747.952004:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e059bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.952013:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e059bc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.952037:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e059bc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.952045:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e059bc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.952136:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493748736:12345-192.168.203.60@tcp:4:dd.0 Request processed in 64805us (65335us total) trans 4294973301 rc 0/0 00000100:00100000:2.0:1789760747.952148:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6012 00000100:00100000:2.0:1789760747.952158:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758181600116 time 758228351397 state 0x2 ref 0 00000100:00100000:1.0:1789760747.960679:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760747.960777:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493748864 00000100:00100000:0.0:1789760747.960787:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493748864 00000100:00100000:0.0:1789760747.960841:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758181600116 time 758228351397 state 0x0 ref 1 00000100:00100000:0.0:1789760747.960856:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758228611813 time 758228351397 state 0x1 ref 1 00000100:00100000:0.0:1789760747.960889:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758228611813 time 758301924665 state 0x0 ref 1 00000100:00100000:0.0:1789760747.960901:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493748864:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760747.960992:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24506368, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc007 00010000:00010000:0.0:1789760747.961001:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e059c3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.961016:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760747.961020:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.961027:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e059c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.961041:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e059c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760748.040314:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973302 is committed 00002000:00010000:2.0:1789760748.040400:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24506368 -> 24510464 00010000:00010000:2.0:1789760748.040420:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e059c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.040445:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e059c3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.040456:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e059c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.040464:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760748.040469:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e059c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.040482:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e059c3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.040502:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e059c3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.040510:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e059c3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.040605:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493748864:12345-192.168.203.60@tcp:4:dd.0 Request processed in 79701us (79931us total) trans 4294973302 rc 0/0 00000100:00100000:2.0:1789760748.040616:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6013 00000100:00100000:2.0:1789760748.040627:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758228611813 time 758301924665 state 0x2 ref 0 00000100:00100000:3.0:1789760748.049319:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760748.049404:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493748992 00000100:00100000:0.0:1789760748.049414:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493748992 00000100:00100000:0.0:1789760748.049479:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758228611813 time 758301924665 state 0x0 ref 1 00000100:00100000:0.0:1789760748.049504:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758302185081 time 758301924665 state 0x1 ref 1 00000100:00100000:0.0:1789760748.049598:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758302185081 time 758390632303 state 0x0 ref 1 00000100:00100000:0.0:1789760748.049611:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493748992:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760748.049662:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24510464, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc007 00010000:00010000:0.0:1789760748.049671:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e059ca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.049687:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760748.049691:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.049698:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e059ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.049712:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e059ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760748.084244:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973303 is committed 00002000:00010000:2.0:1789760748.084447:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24510464 -> 24514560 00010000:00010000:2.0:1789760748.084462:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e059ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.084474:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e059ca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.084493:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e059ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.084501:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760748.084507:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e059ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.084517:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e059ca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.084535:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e059ca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.084543:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e059ca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.084635:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493748992:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35021us (35320us total) trans 4294973303 rc 0/0 00000100:00100000:2.0:1789760748.084645:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6014 00000100:00100000:2.0:1789760748.084655:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758302185081 time 758390632303 state 0x2 ref 0 00010000:00010000:1.0:1789760748.091298:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760748.092184:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760748.092303:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493749120 00000100:00100000:2.0:1789760748.092313:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493749120 00000100:00100000:2.0:1789760748.092375:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758302185081 time 758390632303 state 0x0 ref 1 00000100:00100000:2.0:1789760748.092392:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758390892719 time 758390632303 state 0x1 ref 1 00000100:00100000:2.0:1789760748.092424:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758390892719 time 758433459765 state 0x0 ref 1 00000100:00100000:2.0:1789760748.092448:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493749120:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760748.092501:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24514560, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc007 00010000:00010000:2.0:1789760748.092520:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e059d1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.092537:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760748.092542:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.092549:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e059d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.092563:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e059d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.125405:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973304 is committed 00002000:00010000:2.0:1789760748.125561:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24514560 -> 24518656 00010000:00010000:2.0:1789760748.125577:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e059d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.125589:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e059d1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.125597:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e059d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.125605:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760748.125610:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e059d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.125618:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e059d1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.125643:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e059d1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.125651:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e059d1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.125815:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493749120:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33364us (33632us total) trans 4294973304 rc 0/0 00000100:00100000:2.0:1789760748.125893:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6015 00000100:00100000:2.0:1789760748.126267:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758390892719 time 758433459765 state 0x2 ref 0 00000100:00100000:1.0:1789760748.132202:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760748.132287:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493749248 00000100:00100000:3.0:1789760748.132297:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493749248 00000100:00100000:3.0:1789760748.132355:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758390892719 time 758433459765 state 0x0 ref 1 00000100:00100000:3.0:1789760748.132371:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758433720181 time 758433459765 state 0x1 ref 1 00000100:00100000:3.0:1789760748.132403:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758433720181 time 758473438574 state 0x0 ref 1 00000100:00100000:3.0:1789760748.132424:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493749248:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760748.132491:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24518656, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc007 00010000:00010000:3.0:1789760748.132501:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e059d8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.132522:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760748.132527:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.132543:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e059d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.132558:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e059d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.170140:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973305 is committed 00002000:00010000:0.0:1789760748.170396:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24518656 -> 24522752 00010000:00010000:0.0:1789760748.170412:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e059d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.170426:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e059d8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.170444:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e059d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.170457:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760748.170462:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e059d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.170472:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e059d8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.170550:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e059d8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.170558:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e059d8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.170763:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493749248:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38336us (38566us total) trans 4294973305 rc 0/0 00000100:00100000:0.0:1789760748.170775:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6016 00000100:00100000:0.0:1789760748.170785:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758433720181 time 758473438574 state 0x2 ref 0 00000100:00100000:3.0:1789760748.175314:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760748.175888:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493749376 00000100:00100000:0.0:1789760748.175899:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493749376 00000100:00100000:0.0:1789760748.176182:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758433720181 time 758473438574 state 0x0 ref 1 00000100:00100000:0.0:1789760748.176196:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758473698990 time 758473438574 state 0x1 ref 1 00000100:00100000:0.0:1789760748.176281:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758473698990 time 758517315725 state 0x0 ref 1 00000100:00100000:0.0:1789760748.176298:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493749376:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760748.176346:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24522752, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc007 00010000:00010000:0.0:1789760748.176355:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e059df lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.176372:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760748.176376:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.176383:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e059df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.176397:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e059df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760748.219026:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24522752 -> 24526848 00010000:00010000:2.0:1789760748.219046:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e059df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.219061:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e059df lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.219070:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e059df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.219078:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760748.219082:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e059df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.219096:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e059df lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.219117:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e059df lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.219125:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e059df lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.219212:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493749376:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42912us (43902us total) trans 4294973306 rc 0/0 00000100:00100000:2.0:1789760748.219222:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6017 00000100:00100000:2.0:1789760748.219231:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758473698990 time 758517315725 state 0x2 ref 0 00000001:00080000:1.0:1789760748.219357:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973306 is committed 00000100:00100000:0.0:1789760748.223333:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760748.223500:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493749504 00000100:00100000:1.0:1789760748.223511:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493749504 00000100:00100000:1.0:1789760748.223565:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758473698990 time 758517315725 state 0x0 ref 1 00000100:00100000:1.0:1789760748.223581:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758517576141 time 758517315725 state 0x1 ref 1 00000100:00100000:1.0:1789760748.223645:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758517576141 time 758564680392 state 0x0 ref 1 00000100:00100000:1.0:1789760748.223728:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493749504:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760748.223780:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24526848, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc007 00010000:00010000:1.0:1789760748.223789:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e059e6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.223804:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760748.223809:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.223815:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e059e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.223829:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e059e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760748.271593:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973307 is committed 00002000:00010000:2.0:1789760748.271781:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24526848 -> 24530944 00010000:00010000:2.0:1789760748.271796:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e059e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.271809:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e059e6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.271818:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e059e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.271825:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760748.271831:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e059e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.271840:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e059e6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.271860:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e059e6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.271868:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e059e6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.272033:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493749504:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48373us (48706us total) trans 4294973307 rc 0/0 00000100:00100000:2.0:1789760748.272043:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6018 00000100:00100000:2.0:1789760748.272051:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758517576141 time 758564680392 state 0x2 ref 0 00000100:00100000:0.0:1789760748.276990:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760748.277062:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493749632 00000100:00100000:2.0:1789760748.277072:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493749632 00000100:00100000:2.0:1789760748.277129:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758517576141 time 758564680392 state 0x0 ref 1 00000100:00100000:2.0:1789760748.277144:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758564940808 time 758564680392 state 0x1 ref 1 00000100:00100000:2.0:1789760748.277177:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758564940808 time 758618212709 state 0x0 ref 1 00000100:00100000:2.0:1789760748.277189:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493749632:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760748.277236:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24530944, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc007 00010000:00010000:2.0:1789760748.277246:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e059ed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.277262:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760748.277267:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.277274:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e059ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.277294:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e059ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.337849:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973308 is committed 00002000:00010000:2.0:1789760748.337989:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24530944 -> 24535040 00010000:00010000:2.0:1789760748.338004:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e059ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.338018:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e059ed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.338027:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e059ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.338035:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760748.338040:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e059ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.338048:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e059ed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.338074:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e059ed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.338081:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e059ed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.338166:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493749632:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60973us (61211us total) trans 4294973308 rc 0/0 00000100:00100000:2.0:1789760748.338177:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6019 00000100:00100000:2.0:1789760748.338186:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758564940808 time 758618212709 state 0x2 ref 0 00000100:00100000:3.0:1789760748.344137:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760748.344258:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493749760 00000100:00100000:0.0:1789760748.344269:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493749760 00000100:00100000:0.0:1789760748.344333:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758564940808 time 758618212709 state 0x0 ref 1 00000100:00100000:0.0:1789760748.344356:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758618473125 time 758618212709 state 0x1 ref 1 00000100:00100000:0.0:1789760748.344447:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758618473125 time 758685480765 state 0x0 ref 1 00000100:00100000:0.0:1789760748.345139:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493749760:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760748.345199:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24535040, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc007 00010000:00010000:0.0:1789760748.345210:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e059f4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.345226:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760748.345231:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.345238:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e059f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.345567:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e059f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.388752:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973309 is committed 00002000:00010000:0.0:1789760748.388932:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24535040 -> 24539136 00010000:00010000:0.0:1789760748.388947:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e059f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.388976:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e059f4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.388987:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e059f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.388995:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760748.389000:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e059f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.389010:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e059f4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.389029:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e059f4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.389037:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e059f4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.389158:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493749760:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44695us (45030us total) trans 4294973309 rc 0/0 00000100:00100000:0.0:1789760748.389168:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6020 00000100:00100000:0.0:1789760748.389178:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758618473125 time 758685480765 state 0x2 ref 0 00000100:00100000:0.0:1789760748.393742:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760748.395705:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493749888 00000100:00100000:1.0:1789760748.395718:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493749888 00000100:00100000:1.0:1789760748.395778:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758618473125 time 758685480765 state 0x0 ref 1 00000100:00100000:1.0:1789760748.395794:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758685741181 time 758685480765 state 0x1 ref 1 00000100:00100000:1.0:1789760748.395826:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758685741181 time 758736861257 state 0x0 ref 1 00000100:00100000:1.0:1789760748.395840:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493749888:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760748.395892:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24539136, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc007 00010000:00010000:1.0:1789760748.395901:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e059fb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.395916:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760748.395921:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.395927:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e059fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.395941:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e059fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.412333:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760748.412356:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc33eb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3304:1876699519253888:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760748.412419:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760748.412502:0:6691:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519253888 00000100:00100000:3.0:1789760748.412511:0:6691:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519253888 00000100:00100000:3.0:1789760748.412569:0:6691:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 202 00000100:00100000:3.0:1789760748.412578:0:6691:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519253888:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789760748.412600:0:6691:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527761408 pending 1703936 free 3833593856 avail 3791650816 00000100:00100000:3.0:1789760748.412643:0:6691:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519253888:12345-0@lo:13:osp-pre-0-0.0 Request processed in 66us (223us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760748.412651:0:6691:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 202 00000100:00080000:3.0:1789760748.412670:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760748.412706:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc33eb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3304:1876699519253888:0@lo:13:osp-pre-0-0.0 00000001:00080000:0.0:1789760748.429830:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973310 is committed 00002000:00010000:1.0:1789760748.429993:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24539136 -> 24543232 00010000:00010000:1.0:1789760748.430008:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e059fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.430021:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e059fb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.430030:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e059fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.430048:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760748.430054:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e059fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.430063:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e059fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.430374:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e059fb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.430384:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e059fb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.430742:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493749888:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34900us (37004us total) trans 4294973310 rc 0/0 00000100:00100000:1.0:1789760748.430757:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6021 00000100:00100000:1.0:1789760748.430766:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758685741181 time 758736861257 state 0x2 ref 0 00000100:00100000:1.0:1789760748.434502:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760748.434606:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493750016 00000100:00100000:2.0:1789760748.434615:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493750016 00000100:00100000:2.0:1789760748.434668:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758685741181 time 758736861257 state 0x0 ref 1 00000100:00100000:2.0:1789760748.434683:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758737121673 time 758736861257 state 0x1 ref 1 00000100:00100000:2.0:1789760748.434725:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758737121673 time 758775760852 state 0x0 ref 1 00000100:00100000:2.0:1789760748.434738:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493750016:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760748.434788:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24543232, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc007 00010000:00010000:2.0:1789760748.434797:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05a02 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.434812:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760748.434830:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.434836:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05a02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.434851:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05a02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.479449:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973311 is committed 00002000:00010000:0.0:1789760748.479813:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24543232 -> 24547328 00010000:00010000:0.0:1789760748.479830:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05a02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.479843:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05a02 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.479852:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05a02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.479860:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760748.479864:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05a02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.479873:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05a02 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.479891:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05a02 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.479899:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05a02 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.480029:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493750016:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45288us (45531us total) trans 4294973311 rc 0/0 00000100:00100000:0.0:1789760748.480039:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6022 00000100:00100000:0.0:1789760748.480049:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758737121673 time 758775760852 state 0x2 ref 0 00000100:00100000:3.0:1789760748.487040:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760748.487307:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493750144 00000100:00100000:2.0:1789760748.487317:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493750144 00000100:00100000:2.0:1789760748.487375:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758737121673 time 758775760852 state 0x0 ref 1 00000100:00100000:2.0:1789760748.487390:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758776021268 time 758775760852 state 0x1 ref 1 00000100:00100000:2.0:1789760748.487478:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758776021268 time 758828512219 state 0x0 ref 1 00000100:00100000:2.0:1789760748.487493:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493750144:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760748.487539:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24547328, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc007 00010000:00010000:2.0:1789760748.487547:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05a09 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.487562:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760748.487566:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.487573:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05a09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.487587:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05a09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.534330:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973312 is committed 00002000:00010000:0.0:1789760748.534415:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24547328 -> 24551424 00010000:00010000:0.0:1789760748.534443:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05a09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.534456:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05a09 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.534465:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05a09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.534472:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760748.534477:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05a09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.534485:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05a09 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.534503:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05a09 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.534511:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05a09 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.534933:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493750144:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47438us (47900us total) trans 4294973312 rc 0/0 00000100:00100000:0.0:1789760748.535066:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6023 00000100:00100000:0.0:1789760748.535075:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758776021268 time 758828512219 state 0x2 ref 0 00000100:00100000:2.0:1789760748.541372:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760748.541504:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493750272 00000100:00100000:0.0:1789760748.541513:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493750272 00000100:00100000:0.0:1789760748.541642:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758776021268 time 758828512219 state 0x0 ref 1 00000100:00100000:0.0:1789760748.541658:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758828772635 time 758828512219 state 0x1 ref 1 00000100:00100000:0.0:1789760748.541724:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758828772635 time 758882758413 state 0x0 ref 1 00000100:00100000:0.0:1789760748.541737:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493750272:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760748.541783:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24551424, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc007 00010000:00010000:0.0:1789760748.541791:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05a10 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.541805:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760748.541810:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.541816:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05a10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.541829:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05a10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760748.581092:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24551424 -> 24555520 00010000:00010000:0.0:1789760748.581112:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05a10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.581125:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05a10 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.581134:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05a10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.581142:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760748.581147:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05a10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.581168:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05a10 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.581193:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05a10 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.581201:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e05a10 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.581297:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493750272:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39558us (39930us total) trans 4294973313 rc 0/0 00000100:00100000:0.0:1789760748.581308:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6024 00000100:00100000:0.0:1789760748.581317:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758828772635 time 758882758413 state 0x2 ref 0 00000001:00080000:2.0:1789760748.582627:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973313 is committed 00000100:00100000:1.0:1789760748.586502:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760748.586636:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493750400 00000100:00100000:2.0:1789760748.586646:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493750400 00000100:00100000:2.0:1789760748.586702:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758828772635 time 758882758413 state 0x0 ref 1 00000100:00100000:2.0:1789760748.586716:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758883018829 time 758882758413 state 0x1 ref 1 00000100:00100000:2.0:1789760748.586745:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758883018829 time 758927780543 state 0x0 ref 1 00000100:00100000:2.0:1789760748.586756:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493750400:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760748.586801:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24555520, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xc007 00010000:00010000:2.0:1789760748.586809:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a17 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.586823:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760748.586828:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.586834:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.586846:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.611530:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973314 is committed 00002000:00010000:2.0:1789760748.611686:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24555520 -> 24559616 00002000:00010000:2.0:1789760748.611691:0:13038:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760747 -> 1789760748 00002000:00010000:2.0:1789760748.611694:0:13038:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760747 -> 1789760748 00010000:00010000:2.0:1789760748.611706:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.611719:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a17 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.611729:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a17 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.611736:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760748.611750:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a17 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.611761:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a17 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.611780:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a17 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.611787:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a17 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.611872:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493750400:12345-192.168.203.60@tcp:4:dd.0 Request processed in 25113us (25374us total) trans 4294973314 rc 0/0 00000100:00100000:2.0:1789760748.611883:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6025 00000100:00100000:2.0:1789760748.611892:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758883018829 time 758927780543 state 0x2 ref 0 00000100:00100000:0.0:1789760748.618349:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760748.618563:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493750528 00000100:00100000:2.0:1789760748.618573:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493750528 00000100:00100000:2.0:1789760748.618629:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758883018829 time 758927780543 state 0x0 ref 1 00000100:00100000:2.0:1789760748.618656:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758928040959 time 758927780543 state 0x1 ref 1 00000100:00100000:2.0:1789760748.618721:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758928040959 time 758959756154 state 0x0 ref 1 00000100:00100000:2.0:1789760748.618735:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493750528:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760748.618794:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24559616, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:2.0:1789760748.618803:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a1e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.618819:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760748.618824:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.618830:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.618844:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760748.649955:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24559616 -> 24563712 00010000:00010000:2.0:1789760748.649997:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.650010:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a1e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.650019:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.650026:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760748.650031:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.650040:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a1e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.650058:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a1e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.650066:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a1e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.650162:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493750528:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31425us (31817us total) trans 4294973315 rc 0/0 00000100:00100000:2.0:1789760748.650173:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6026 00000100:00100000:2.0:1789760748.650183:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758928040959 time 758959756154 state 0x2 ref 0 00000001:00080000:1.0:1789760748.651272:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973315 is committed 00000100:00100000:0.0:1789760748.658365:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760748.658466:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493750656 00000100:00100000:2.0:1789760748.658474:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493750656 00000100:00100000:2.0:1789760748.658531:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758928040959 time 758959756154 state 0x0 ref 1 00000100:00100000:2.0:1789760748.658546:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758960016570 time 758959756154 state 0x1 ref 1 00000100:00100000:2.0:1789760748.658576:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758960016570 time 758999612306 state 0x0 ref 1 00000100:00100000:2.0:1789760748.658589:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493750656:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760748.658635:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24563712, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:2.0:1789760748.658644:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a25 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.658659:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760748.658663:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.658669:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.658683:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.695450:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973316 is committed 00002000:00010000:2.0:1789760748.695766:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24563712 -> 24567808 00010000:00010000:2.0:1789760748.695782:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.695795:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a25 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.695804:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.695811:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760748.695816:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.695825:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a25 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.695845:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a25 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.695853:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05a25 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.696007:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493750656:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37417us (37647us total) trans 4294973316 rc 0/0 00000100:00100000:2.0:1789760748.696019:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6027 00000100:00100000:2.0:1789760748.696029:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758960016570 time 758999612306 state 0x2 ref 0 00000100:00100000:0.0:1789760748.703214:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760748.703349:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493750784 00000100:00100000:3.0:1789760748.703359:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493750784 00000100:00100000:3.0:1789760748.703412:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758960016570 time 758999612306 state 0x0 ref 1 00000100:00100000:3.0:1789760748.703442:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758999872722 time 758999612306 state 0x1 ref 1 00000100:00100000:3.0:1789760748.703472:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758999872722 time 759044508472 state 0x0 ref 1 00000100:00100000:3.0:1789760748.703594:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493750784:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760748.703643:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24567808, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:3.0:1789760748.703652:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05a2c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.703668:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760748.703673:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.703679:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05a2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.703691:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05a2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760748.744335:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973317 is committed 00002000:00010000:1.0:1789760748.744470:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24567808 -> 24571904 00010000:00010000:1.0:1789760748.744493:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05a2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.744505:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05a2c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.744513:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05a2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.744520:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760748.744525:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05a2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.744532:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05a2c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.744557:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05a2c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.744563:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05a2c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.744651:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493750784:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41163us (41441us total) trans 4294973317 rc 0/0 00000100:00100000:1.0:1789760748.744661:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6028 00000100:00100000:1.0:1789760748.744670:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758999872722 time 759044508472 state 0x2 ref 0 00000100:00100000:0.0:1789760748.748644:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760748.748786:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493750912 00000100:00100000:1.0:1789760748.748795:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493750912 00000100:00100000:1.0:1789760748.748848:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 758999872722 time 759044508472 state 0x0 ref 1 00000100:00100000:1.0:1789760748.748862:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759044768888 time 759044508472 state 0x1 ref 1 00000100:00100000:1.0:1789760748.748935:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759044768888 time 759089970433 state 0x0 ref 1 00000100:00100000:1.0:1789760748.748948:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493750912:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760748.749038:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24571904, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:1.0:1789760748.749046:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a33 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.749061:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760748.749065:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.749071:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.749083:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760748.789247:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973318 is committed 00002000:00010000:1.0:1789760748.789374:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24571904 -> 24576000 00010000:00010000:1.0:1789760748.789391:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.789403:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a33 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.789412:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.789419:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760748.789433:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.789442:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a33 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.789462:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a33 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.789474:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a33 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.789586:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493750912:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40634us (40944us total) trans 4294973318 rc 0/0 00000100:00100000:1.0:1789760748.789606:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6029 00000100:00100000:1.0:1789760748.789616:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759044768888 time 759089970433 state 0x2 ref 0 00000100:00100000:2.0:1789760748.795362:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760748.795598:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493751040 00000100:00100000:0.0:1789760748.795609:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493751040 00000100:00100000:0.0:1789760748.795665:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759044768888 time 759089970433 state 0x0 ref 1 00000100:00100000:0.0:1789760748.795681:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759090230849 time 759089970433 state 0x1 ref 1 00000100:00100000:0.0:1789760748.795712:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759090230849 time 759136748033 state 0x0 ref 1 00000100:00100000:0.0:1789760748.795725:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493751040:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760748.795776:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24576000, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:0.0:1789760748.795785:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05a3a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.795802:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760748.795806:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.795812:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05a3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.795825:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05a3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760748.859381:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973319 is committed 00002000:00010000:1.0:1789760748.859526:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24576000 -> 24580096 00010000:00010000:1.0:1789760748.859541:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05a3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.859555:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05a3a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.859564:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05a3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.859572:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760748.859577:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05a3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.859587:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05a3a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.859612:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05a3a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.859619:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05a3a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.859715:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493751040:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63987us (64361us total) trans 4294973319 rc 0/0 00000100:00100000:1.0:1789760748.859725:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6030 00000100:00100000:1.0:1789760748.859735:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759090230849 time 759136748033 state 0x2 ref 0 00000100:00100000:0.0:1789760748.865005:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760748.866902:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493751168 00000100:00100000:2.0:1789760748.866915:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493751168 00000100:00100000:3.0:1789760748.867304:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759090230849 time 759136748033 state 0x0 ref 1 00000100:00100000:3.0:1789760748.867321:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759137008449 time 759136748033 state 0x1 ref 1 00000100:00100000:3.0:1789760748.867358:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759137008449 time 759208393382 state 0x0 ref 1 00000100:00100000:3.0:1789760748.867371:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493751168:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760748.867420:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24580096, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:3.0:1789760748.867442:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05a41 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.867458:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760748.867462:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.867469:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05a41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.867482:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05a41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760748.900460:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973320 is committed 00002000:00010000:1.0:1789760748.900834:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24580096 -> 24584192 00010000:00010000:1.0:1789760748.900850:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05a41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.900864:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05a41 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.900873:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05a41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.900880:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760748.900885:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05a41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.900899:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05a41 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.900918:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05a41 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.900926:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05a41 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.901249:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493751168:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33875us (36249us total) trans 4294973320 rc 0/0 00000100:00100000:1.0:1789760748.901261:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6031 00000100:00100000:1.0:1789760748.901271:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759137008449 time 759208393382 state 0x2 ref 0 00000100:00100000:2.0:1789760748.903713:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760748.903796:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493751296 00000100:00100000:1.0:1789760748.903806:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493751296 00000100:00100000:1.0:1789760748.903860:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759137008449 time 759208393382 state 0x0 ref 1 00000100:00100000:1.0:1789760748.903875:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759208653798 time 759208393382 state 0x1 ref 1 00000100:00100000:1.0:1789760748.903908:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759208653798 time 759244944173 state 0x0 ref 1 00000100:00100000:1.0:1789760748.903921:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493751296:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760748.904037:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24584192, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:1.0:1789760748.904047:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a48 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.904064:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760748.904068:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.904074:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.904089:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.922617:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff962101a35500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3304:1876699519254016:0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.922690:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760748.922742:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4360700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3304:1876699519254528:0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.922775:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760748.922806:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760748.922812:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4362d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3304:1876699519254400:0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.922834:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760748.922860:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4d88700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519254272:0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.922883:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760748.922909:0:3306:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1789760748.922914:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff962101a35f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3306:1876699519254144:0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.922938:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760748.922956:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620ffc15500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519254656:0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.923008:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760748.923042:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519254016 00000100:00100000:0.0:1789760748.923049:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519254016 00000100:00100000:0.0:1789760748.923098:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 402 00000100:00100000:0.0:1789760748.923106:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d49c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3304:x1876699519254016:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.923148:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d49c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3304:x1876699519254016:12345-0@lo:400:kworker.0 Request processed in 43us (460us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760748.923155:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 402 00000100:00100000:0.0:1789760748.923183:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519254528 00000100:00100000:0.0:1789760748.923187:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519254528 00000100:00100000:0.0:1789760748.923212:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519254400 00000100:00100000:0.0:1789760748.923214:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519254400 00000100:00100000:0.0:1789760748.923229:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519254144 00000100:00100000:0.0:1789760748.923231:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519254144 00000100:00100000:0.0:1789760748.923242:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 489 00000100:00100000:0.0:1789760748.923246:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc17480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3304:x1876699519254528:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.923274:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc17480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3304:x1876699519254528:12345-0@lo:400:kworker.0 Request processed in 27us (499us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760748.923279:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 489 00000100:00100000:0.0:1789760748.923289:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 490 00000100:00100000:0.0:1789760748.923295:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc16300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3304:x1876699519254400:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.923321:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc16300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3304:x1876699519254400:12345-0@lo:400:kworker.0 Request processed in 25us (484us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760748.923327:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 490 00000100:00080000:3.0:1789760748.923333:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760748.923344:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 491 00000100:00100000:0.0:1789760748.923352:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3306:x1876699519254144:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.923403:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3306:x1876699519254144:12345-0@lo:400:kworker.0 Request processed in 52us (463us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760748.923408:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 491 00000100:00100000:0.0:1789760748.923444:0:13040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519254272 00000100:00100000:0.0:1789760748.923447:0:13040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519254272 00000100:00100000:0.0:1789760748.923473:0:13040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519254656 00000100:00100000:0.0:1789760748.923476:0:13040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519254656 00000100:00100000:0.0:1789760748.923509:0:13040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 176 00000100:00100000:0.0:1789760748.923516:0:13040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519254272:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.923554:0:13040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519254272:12345-0@lo:400:kworker.0 Request processed in 38us (669us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760748.923561:0:13040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 176 00000100:00100000:0.0:1789760748.923579:0:13040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 177 00000100:00100000:0.0:1789760748.923588:0:13040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519254656:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.923614:0:13040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519254656:12345-0@lo:400:kworker.0 Request processed in 30us (607us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760748.923619:0:13040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 177 00000100:00080000:0.0:1789760748.923656:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789760748.923683:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff962101a35f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3306:1876699519254144:0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.923694:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620ffc15500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519254656:0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.923703:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4d88700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519254272:0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.923733:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4362d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3304:1876699519254400:0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.923740:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff962101a35500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3304:1876699519254016:0@lo:400:kworker.0 00000100:00100000:0.0:1789760748.923752:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4360700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3304:1876699519254528:0@lo:400:kworker.0 00002000:00010000:1.0:1789760748.941486:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24584192 -> 24588288 00010000:00010000:1.0:1789760748.941507:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.941520:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a48 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.941529:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.941537:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760748.941541:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.941551:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a48 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.941570:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a48 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.941578:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a48 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.941674:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493751296:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37750us (37965us total) trans 4294973321 rc 0/0 00000100:00100000:1.0:1789760748.941685:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6032 00000100:00100000:1.0:1789760748.941695:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759208653798 time 759244944173 state 0x2 ref 0 00000001:00080000:0.0:1789760748.942223:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973321 is committed 00000100:00100000:0.0:1789760748.951081:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760748.951178:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493751424 00000100:00100000:1.0:1789760748.951188:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493751424 00000100:00100000:1.0:1789760748.951244:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759208653798 time 759244944173 state 0x0 ref 1 00000100:00100000:1.0:1789760748.951263:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759245204589 time 759244944173 state 0x1 ref 1 00000100:00100000:1.0:1789760748.951295:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759245204589 time 759292330328 state 0x0 ref 1 00000100:00100000:1.0:1789760748.951308:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493751424:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760748.951355:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24588288, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:1.0:1789760748.951364:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a4f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.951378:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760748.951382:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.951388:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.951407:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760748.990051:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24588288 -> 24592384 00010000:00010000:1.0:1789760748.990071:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.990084:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a4f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.990094:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.990103:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760748.990108:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.990118:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a4f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.990137:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a4f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.990145:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a4f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.990247:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493751424:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38937us (39171us total) trans 4294973322 rc 0/0 00000100:00100000:1.0:1789760748.990256:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6033 00000100:00100000:1.0:1789760748.990266:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759245204589 time 759292330328 state 0x2 ref 0 00000001:00080000:3.0:1789760748.990833:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973322 is committed 00000100:00100000:3.0:1789760748.998706:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760748.998798:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493751552 00000100:00100000:0.0:1789760748.998807:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493751552 00000100:00100000:0.0:1789760748.998859:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759245204589 time 759292330328 state 0x0 ref 1 00000100:00100000:0.0:1789760748.998873:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759292590744 time 759292330328 state 0x1 ref 1 00000100:00100000:0.0:1789760748.998904:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759292590744 time 759339940217 state 0x0 ref 1 00000100:00100000:0.0:1789760748.998916:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493751552:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760748.998998:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24592384, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:0.0:1789760748.999008:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05a56 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.999023:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760748.999028:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.999034:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05a56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.999048:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05a56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760749.027706:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973323 is committed 00002000:00010000:0.0:1789760749.027993:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24592384 -> 24596480 00010000:00010000:0.0:1789760749.028008:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05a56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.028021:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05a56 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.028031:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05a56 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.028039:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760749.028043:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05a56 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.028053:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05a56 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.028072:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05a56 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.028080:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05a56 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.028347:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493751552:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29428us (29646us total) trans 4294973323 rc 0/0 00000100:00100000:0.0:1789760749.028358:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6034 00000100:00100000:0.0:1789760749.028367:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759292590744 time 759339940217 state 0x2 ref 0 00000100:00100000:0.0:1789760749.036936:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760749.037102:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493751680 00000100:00100000:3.0:1789760749.037113:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493751680 00000100:00100000:3.0:1789760749.037187:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759292590744 time 759339940217 state 0x0 ref 1 00000100:00100000:3.0:1789760749.037201:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759340200633 time 759339940217 state 0x1 ref 1 00000100:00100000:3.0:1789760749.037233:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759340200633 time 759378268881 state 0x0 ref 1 00000100:00100000:3.0:1789760749.037247:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493751680:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760749.037298:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24596480, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:3.0:1789760749.037307:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05a5d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.037323:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760749.037327:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.037346:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05a5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.037361:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05a5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760749.037775:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789760749.091416:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24596480 -> 24600576 00010000:00010000:0.0:1789760749.091451:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05a5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.091465:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05a5d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.091483:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05a5d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.091490:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760749.091495:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05a5d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.091505:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05a5d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.091524:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05a5d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.091532:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05a5d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.091627:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493751680:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54378us (54697us total) trans 4294973324 rc 0/0 00000100:00100000:0.0:1789760749.091639:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6035 00000100:00100000:0.0:1789760749.091648:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759340200633 time 759378268881 state 0x2 ref 0 00000001:00080000:1.0:1789760749.092184:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973324 is committed 00000100:00100000:2.0:1789760749.102782:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760749.102904:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493751808 00000100:00100000:1.0:1789760749.102913:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493751808 00000100:00100000:1.0:1789760749.103323:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759340200633 time 759378268881 state 0x0 ref 1 00000100:00100000:1.0:1789760749.103339:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759378529297 time 759378268881 state 0x1 ref 1 00000100:00100000:2.0:1789760749.103666:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759378529297 time 759444698897 state 0x0 ref 1 00000100:00100000:2.0:1789760749.103683:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493751808:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760749.103734:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24600576, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:2.0:1789760749.103744:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05a64 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.103760:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:2.0:1789760749.103764:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.103771:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05a64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.103785:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05a64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.114577:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760749.137597:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973325 is committed 00002000:00010000:0.0:1789760749.137928:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24600576 -> 24604672 00010000:00010000:0.0:1789760749.137944:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05a64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.137958:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05a64 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.138115:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05a64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.138124:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760749.138129:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05a64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.138139:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05a64 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.138160:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05a64 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.138167:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05a64 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.138308:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493751808:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34626us (35531us total) trans 4294973325 rc 0/0 00000100:00100000:0.0:1789760749.138326:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6036 00000100:00100000:0.0:1789760749.138336:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759378529297 time 759444698897 state 0x2 ref 0 00000100:00100000:2.0:1789760749.141821:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760749.142868:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493751936 00000100:00100000:1.0:1789760749.142879:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493751936 00000100:00100000:1.0:1789760749.142934:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759378529297 time 759444698897 state 0x0 ref 1 00000100:00100000:1.0:1789760749.142949:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759444959313 time 759444698897 state 0x1 ref 1 00000100:00100000:1.0:1789760749.143084:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759444959313 time 759484118497 state 0x0 ref 1 00000100:00100000:1.0:1789760749.143098:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493751936:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760749.143209:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24604672, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:1.0:1789760749.143218:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a6b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.143236:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760749.143240:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.143250:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.143264:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760749.175066:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973326 is committed 00002000:00010000:3.0:1789760749.175322:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24604672 -> 24608768 00010000:00010000:3.0:1789760749.175336:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.175349:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a6b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.175359:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.175366:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760749.175371:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.175381:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a6b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.175400:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a6b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.175408:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05a6b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.175675:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493751936:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32574us (33857us total) trans 4294973326 rc 0/0 00000100:00100000:3.0:1789760749.175687:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6037 00000100:00100000:3.0:1789760749.175697:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759444959313 time 759484118497 state 0x2 ref 0 00000100:00100000:2.0:1789760749.181691:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760749.181846:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493752064 00000100:00100000:0.0:1789760749.181855:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493752064 00000100:00100000:0.0:1789760749.181916:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759444959313 time 759484118497 state 0x0 ref 1 00000100:00100000:0.0:1789760749.181932:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759484378913 time 759484118497 state 0x1 ref 1 00000100:00100000:0.0:1789760749.181993:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759484378913 time 759522997551 state 0x0 ref 1 00000100:00100000:0.0:1789760749.182006:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493752064:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760749.182437:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24608768, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:1.0:1789760749.182448:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05a72 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.182463:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760749.182467:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.182535:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05a72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.182553:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05a72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.216831:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973327 is committed 00002000:00010000:0.0:1789760749.216957:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24608768 -> 24612864 00010000:00010000:0.0:1789760749.216993:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05a72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.217006:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05a72 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.217015:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05a72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.217023:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760749.217028:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05a72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.217043:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05a72 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.217062:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05a72 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.217070:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05a72 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.217172:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493752064:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35163us (35486us total) trans 4294973327 rc 0/0 00000100:00100000:0.0:1789760749.217191:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6038 00000100:00100000:0.0:1789760749.217201:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759484378913 time 759522997551 state 0x2 ref 0 00000100:00100000:2.0:1789760749.222922:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760749.223262:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493752192 00000100:00100000:0.0:1789760749.223273:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493752192 00000100:00100000:0.0:1789760749.223337:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759484378913 time 759522997551 state 0x0 ref 1 00000100:00100000:0.0:1789760749.223352:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759523257967 time 759522997551 state 0x1 ref 1 00000100:00100000:0.0:1789760749.223438:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759523257967 time 759564472403 state 0x0 ref 1 00000100:00100000:0.0:1789760749.223452:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493752192:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760749.223514:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24612864, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:0.0:1789760749.223522:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05a79 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.223538:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760749.223543:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.223548:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05a79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.223561:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05a79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760749.252808:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973328 is committed 00002000:00010000:3.0:1789760749.252917:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24612864 -> 24616960 00010000:00010000:3.0:1789760749.252934:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05a79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.252946:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05a79 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.252955:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05a79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.252993:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760749.252998:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05a79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.253008:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05a79 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.253027:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05a79 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.253041:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05a79 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.253127:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493752192:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29673us (30221us total) trans 4294973328 rc 0/0 00000100:00100000:3.0:1789760749.253137:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6039 00000100:00100000:3.0:1789760749.256073:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759523257967 time 759564472403 state 0x2 ref 0 00000100:00100000:3.0:1789760749.256899:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760749.256997:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493752320 00000100:00100000:0.0:1789760749.257009:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493752320 00000100:00100000:0.0:1789760749.257062:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759523257967 time 759564472403 state 0x0 ref 1 00000100:00100000:0.0:1789760749.257076:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759564732819 time 759564472403 state 0x1 ref 1 00000100:00100000:0.0:1789760749.257107:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759564732819 time 759598142521 state 0x0 ref 1 00000100:00100000:0.0:1789760749.257119:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493752320:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760749.257181:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24616960, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:0.0:1789760749.257190:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05a80 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.257208:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760749.257213:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.257221:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05a80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.257243:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05a80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760749.293260:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973329 is committed 00002000:00010000:0.0:1789760749.293359:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24616960 -> 24621056 00010000:00010000:0.0:1789760749.293374:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05a80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.293387:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05a80 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.293396:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05a80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.293403:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760749.293408:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05a80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.293417:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05a80 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.293448:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05a80 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.293455:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05a80 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.293558:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493752320:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36436us (36659us total) trans 4294973329 rc 0/0 00000100:00100000:0.0:1789760749.293568:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6040 00000100:00100000:0.0:1789760749.293578:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759564732819 time 759598142521 state 0x2 ref 0 00000100:00100000:0.0:1789760749.297116:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760749.297446:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493752448 00000100:00100000:3.0:1789760749.297456:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493752448 00000100:00100000:3.0:1789760749.297517:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759564732819 time 759598142521 state 0x0 ref 1 00000100:00100000:3.0:1789760749.297532:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759598402937 time 759598142521 state 0x1 ref 1 00000100:00100000:3.0:1789760749.297558:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759598402937 time 759638593877 state 0x0 ref 1 00000100:00100000:3.0:1789760749.297571:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493752448:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760749.297621:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24621056, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:3.0:1789760749.297631:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05a87 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.297646:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760749.297651:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.297657:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05a87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.297673:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05a87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760749.333155:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973330 is committed 00002000:00010000:0.0:1789760749.333330:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24621056 -> 24625152 00010000:00010000:0.0:1789760749.333342:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05a87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.333351:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05a87 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.333357:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05a87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.333362:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760749.333365:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05a87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.333372:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05a87 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.333385:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05a87 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.333393:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05a87 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.333489:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493752448:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35915us (36377us total) trans 4294973330 rc 0/0 00000100:00100000:0.0:1789760749.333499:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6041 00000100:00100000:0.0:1789760749.333509:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759598402937 time 759638593877 state 0x2 ref 0 00000100:00100000:1.0:1789760749.337095:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760749.337164:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493752576 00000100:00100000:0.0:1789760749.337174:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493752576 00000100:00100000:0.0:1789760749.337297:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759598402937 time 759638593877 state 0x0 ref 1 00000100:00100000:0.0:1789760749.337309:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759638854293 time 759638593877 state 0x1 ref 1 00000100:00100000:0.0:1789760749.337333:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759638854293 time 759678368939 state 0x0 ref 1 00000100:00100000:0.0:1789760749.337346:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493752576:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760749.337391:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24625152, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:0.0:1789760749.337399:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05a8e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.337414:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760749.337431:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.337436:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05a8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.337453:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05a8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760749.365283:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24625152 -> 24629248 00010000:00010000:0.0:1789760749.365303:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05a8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.365316:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05a8e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.365324:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05a8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.365331:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760749.365336:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05a8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.365345:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05a8e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.365364:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05a8e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.365372:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05a8e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.365485:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493752576:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28135us (28393us total) trans 4294973331 rc 0/0 00000100:00100000:0.0:1789760749.365495:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6042 00000100:00100000:0.0:1789760749.365506:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759638854293 time 759678368939 state 0x2 ref 0 00000001:00080000:1.0:1789760749.366140:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973331 is committed 00000100:00100000:2.0:1789760749.370092:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760749.370121:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493752704 00000100:00100000:0.0:1789760749.370131:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493752704 00000100:00100000:0.0:1789760749.370187:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759638854293 time 759678368939 state 0x0 ref 1 00000100:00100000:0.0:1789760749.370202:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759678629355 time 759678368939 state 0x1 ref 1 00000100:00100000:0.0:1789760749.370232:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759678629355 time 759711268022 state 0x0 ref 1 00000100:00100000:0.0:1789760749.370245:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493752704:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760749.370298:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24629248, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:0.0:1789760749.370307:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05a95 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.370323:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760749.370327:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.370334:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05a95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.370353:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05a95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760749.408492:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24629248 -> 24633344 00010000:00010000:0.0:1789760749.408512:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05a95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.408526:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05a95 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.408535:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05a95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.408542:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760749.408547:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05a95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.408558:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05a95 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.408576:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05a95 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.408584:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05a95 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.408680:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493752704:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38431us (38592us total) trans 4294973332 rc 0/0 00000100:00100000:0.0:1789760749.408694:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6043 00000100:00100000:0.0:1789760749.408704:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759678629355 time 759711268022 state 0x2 ref 0 00000001:00080000:0.0:1789760749.412042:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973332 is committed 00000100:00100000:3.0:1789760749.412737:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760749.412804:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493752832 00000100:00100000:2.0:1789760749.412814:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493752832 00000100:00100000:2.0:1789760749.412865:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759678629355 time 759711268022 state 0x0 ref 1 00000100:00100000:2.0:1789760749.412879:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759711528438 time 759711268022 state 0x1 ref 1 00000100:00100000:2.0:1789760749.412914:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759711528438 time 759753949860 state 0x0 ref 1 00000100:00100000:2.0:1789760749.412936:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493752832:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760749.413006:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24633344, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:2.0:1789760749.413016:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05a9c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.413032:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:2.0:1789760749.413037:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.413042:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05a9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.413055:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05a9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760749.434328:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760749.434351:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc48dd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519254784:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760749.434427:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760749.434499:0:6691:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519254784 00000100:00100000:1.0:1789760749.434507:0:6691:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519254784 00000100:00100000:1.0:1789760749.434551:0:6691:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 203 00000100:00100000:1.0:1789760749.434560:0:6691:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519254784:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760749.434594:0:6691:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760749.434639:0:6691:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519254784:12345-0@lo:13:osp-pre-1-0.0 Request processed in 80us (221us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760749.434647:0:6691:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 203 00000100:00100000:1.0:1789760749.434691:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc48dd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519254784:0@lo:13:osp-pre-1-0.0 00002000:00010000:0.0:1789760749.465719:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24633344 -> 24637440 00010000:00010000:0.0:1789760749.465738:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05a9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.465751:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05a9c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.465761:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05a9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.465768:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760749.465773:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05a9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.465783:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05a9c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.465810:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05a9c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.465818:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05a9c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.465910:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493752832:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52965us (53169us total) trans 4294973333 rc 0/0 00000100:00100000:0.0:1789760749.465920:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6044 00000100:00100000:0.0:1789760749.465933:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759711528438 time 759753949860 state 0x2 ref 0 00000001:00080000:0.0:1789760749.466001:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973333 is committed 00000100:00100000:3.0:1789760749.478288:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760749.478391:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493752960 00000100:00100000:0.0:1789760749.478400:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493752960 00000100:00100000:0.0:1789760749.478465:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759711528438 time 759753949860 state 0x0 ref 1 00000100:00100000:0.0:1789760749.478480:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759754210276 time 759753949860 state 0x1 ref 1 00000100:00100000:0.0:1789760749.478606:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759754210276 time 759819640269 state 0x0 ref 1 00000100:00100000:0.0:1789760749.478619:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493752960:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760749.478666:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24637440, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:0.0:1789760749.478675:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05aa3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.478691:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760749.478696:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.478702:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05aa3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.478715:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05aa3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.521815:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973334 is committed 00002000:00010000:0.0:1789760749.522109:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24637440 -> 24641536 00010000:00010000:0.0:1789760749.522125:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05aa3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.522200:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05aa3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.522210:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05aa3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.522218:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760749.522223:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05aa3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.522233:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05aa3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.522252:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05aa3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.522259:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05aa3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.522346:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493752960:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43724us (44062us total) trans 4294973334 rc 0/0 00000100:00100000:0.0:1789760749.522360:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6045 00000100:00100000:0.0:1789760749.522369:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759754210276 time 759819640269 state 0x2 ref 0 00000100:00100000:2.0:1789760749.527805:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760749.528460:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493753088 00000100:00100000:0.0:1789760749.528541:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493753088 00000100:00100000:0.0:1789760749.528604:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759754210276 time 759819640269 state 0x0 ref 1 00000100:00100000:0.0:1789760749.528620:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759819900685 time 759819640269 state 0x1 ref 1 00000100:00100000:0.0:1789760749.528717:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759819900685 time 759869682695 state 0x0 ref 1 00000100:00100000:0.0:1789760749.528734:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493753088:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760749.528782:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24641536, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:0.0:1789760749.528791:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05aaa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.528805:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760749.528809:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.528815:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05aaa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.528829:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05aaa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.590078:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973335 is committed 00002000:00010000:0.0:1789760749.590267:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24641536 -> 24645632 00010000:00010000:0.0:1789760749.590281:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05aaa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.590295:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05aaa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.590304:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05aaa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.590311:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760749.590316:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05aaa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.590326:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05aaa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.590344:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05aaa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.590352:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05aaa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.590498:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493753088:12345-192.168.203.60@tcp:4:dd.0 Request processed in 61760us (62694us total) trans 4294973335 rc 0/0 00000100:00100000:0.0:1789760749.590510:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6046 00000100:00100000:0.0:1789760749.590520:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759819900685 time 759869682695 state 0x2 ref 0 00000100:00100000:3.0:1789760749.596282:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760749.596332:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493753216 00000100:00100000:0.0:1789760749.596342:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493753216 00000100:00100000:0.0:1789760749.596401:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759819900685 time 759869682695 state 0x0 ref 1 00000100:00100000:0.0:1789760749.596426:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759869943111 time 759869682695 state 0x1 ref 1 00000100:00100000:0.0:1789760749.596452:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759869943111 time 759937488456 state 0x0 ref 1 00000100:00100000:0.0:1789760749.596465:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493753216:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760749.596513:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24645632, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xc007 00010000:00010000:0.0:1789760749.596522:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ab1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.596536:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760749.596541:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.596548:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ab1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.596562:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ab1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760749.642069:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24645632 -> 24649728 00002000:00010000:0.0:1789760749.642077:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760748 -> 1789760749 00002000:00010000:0.0:1789760749.642080:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760748 -> 1789760749 00010000:00010000:0.0:1789760749.642093:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ab1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.642106:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ab1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.642115:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ab1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.642123:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760749.642128:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ab1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.642137:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ab1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.642157:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ab1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.642165:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ab1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760749.642182:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973336 is committed 00000100:00100000:0.0:1789760749.642579:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493753216:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46110us (46300us total) trans 4294973336 rc 0/0 00000100:00100000:0.0:1789760749.642590:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6047 00000100:00100000:0.0:1789760749.642600:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759869943111 time 759937488456 state 0x2 ref 0 00000100:00100000:3.0:1789760749.651904:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760749.652010:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493753344 00000100:00100000:2.0:1789760749.652020:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493753344 00000100:00100000:2.0:1789760749.652077:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759869943111 time 759937488456 state 0x0 ref 1 00000100:00100000:2.0:1789760749.652092:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759937748872 time 759937488456 state 0x1 ref 1 00000100:00100000:2.0:1789760749.652179:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759937748872 time 759993213218 state 0x0 ref 1 00000100:00100000:2.0:1789760749.652193:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493753344:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760749.652240:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24649728, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:2.0:1789760749.652249:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05ab8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.652264:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=759s v=(0 0) 00010000:00010000:2.0:1789760749.652268:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.652275:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05ab8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.652288:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05ab8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760749.690732:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973337 is committed 00002000:00010000:0.0:1789760749.690878:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24649728 -> 24653824 00010000:00010000:0.0:1789760749.690893:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05ab8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.690906:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05ab8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.690912:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05ab8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.690917:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760749.690921:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05ab8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.690930:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05ab8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.690953:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05ab8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.690988:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05ab8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.691184:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493753344:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38988us (39285us total) trans 4294973337 rc 0/0 00000100:00100000:0.0:1789760749.691196:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6048 00000100:00100000:0.0:1789760749.691206:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759937748872 time 759993213218 state 0x2 ref 0 00000100:00100000:2.0:1789760749.696398:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760749.696561:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493753472 00000100:00100000:3.0:1789760749.696571:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493753472 00000100:00100000:3.0:1789760749.696628:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759937748872 time 759993213218 state 0x0 ref 1 00000100:00100000:3.0:1789760749.696643:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759993473634 time 759993213218 state 0x1 ref 1 00000100:00100000:3.0:1789760749.696706:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759993473634 time 760037740473 state 0x0 ref 1 00000100:00100000:3.0:1789760749.696719:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493753472:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760749.696767:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24653824, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:3.0:1789760749.696776:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05abf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.696793:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760749.696799:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.696806:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05abf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.696824:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05abf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760749.731269:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973338 is committed 00002000:00010000:0.0:1789760749.731407:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24653824 -> 24657920 00010000:00010000:0.0:1789760749.731435:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05abf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.731448:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05abf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.731457:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05abf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.731465:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760749.731480:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05abf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.731491:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05abf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.731511:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05abf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.731520:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05abf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.731614:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493753472:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34890us (35219us total) trans 4294973338 rc 0/0 00000100:00100000:0.0:1789760749.731625:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6049 00000100:00100000:0.0:1789760749.731636:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759993473634 time 760037740473 state 0x2 ref 0 00000100:00100000:3.0:1789760749.745630:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760749.745952:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493753600 00000100:00100000:0.0:1789760749.745997:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493753600 00000100:00100000:0.0:1789760749.746057:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 759993473634 time 760037740473 state 0x0 ref 1 00000100:00100000:0.0:1789760749.746073:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760038000889 time 760037740473 state 0x1 ref 1 00000100:00100000:0.0:1789760749.746110:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760038000889 time 760087146031 state 0x0 ref 1 00000100:00100000:0.0:1789760749.746124:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493753600:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760749.746182:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24657920, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:0.0:1789760749.746191:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ac6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.746208:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760749.746213:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.746219:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ac6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.746231:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ac6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760749.785060:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973339 is committed 00002000:00010000:0.0:1789760749.785150:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24657920 -> 24662016 00010000:00010000:0.0:1789760749.785166:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ac6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.785176:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ac6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.785186:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ac6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.785194:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760749.785199:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ac6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.785208:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ac6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.785228:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ac6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.785236:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ac6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.785336:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493753600:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39209us (39712us total) trans 4294973339 rc 0/0 00000100:00100000:0.0:1789760749.785347:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6050 00000100:00100000:0.0:1789760749.785355:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760038000889 time 760087146031 state 0x2 ref 0 00000100:00100000:1.0:1789760749.791563:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760749.791855:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493753728 00000100:00100000:0.0:1789760749.791865:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493753728 00000100:00100000:0.0:1789760749.791920:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760038000889 time 760087146031 state 0x0 ref 1 00000100:00100000:0.0:1789760749.791935:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760087406447 time 760087146031 state 0x1 ref 1 00000100:00100000:3.0:1789760749.792197:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760087406447 time 760133230023 state 0x0 ref 1 00000100:00100000:3.0:1789760749.792215:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493753728:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760749.792271:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24662016, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:3.0:1789760749.792281:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05acd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.792300:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760749.792305:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.792311:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05acd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.792325:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05acd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760749.827553:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973340 is committed 00002000:00010000:3.0:1789760749.827696:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24662016 -> 24666112 00010000:00010000:3.0:1789760749.827711:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05acd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.827724:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05acd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.827734:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05acd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.827741:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760749.827746:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05acd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.827754:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05acd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.827785:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05acd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.827794:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05acd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.827888:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493753728:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35672us (36333us total) trans 4294973340 rc 0/0 00000100:00100000:3.0:1789760749.827899:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6051 00000100:00100000:3.0:1789760749.827908:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760087406447 time 760133230023 state 0x2 ref 0 00000100:00100000:2.0:1789760749.836597:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760749.836763:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493753856 00000100:00100000:0.0:1789760749.836773:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493753856 00000100:00100000:0.0:1789760749.836827:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760087406447 time 760133230023 state 0x0 ref 1 00000100:00100000:0.0:1789760749.836842:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760133490439 time 760133230023 state 0x1 ref 1 00000100:00100000:0.0:1789760749.836873:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760133490439 time 760177908519 state 0x0 ref 1 00000100:00100000:0.0:1789760749.836885:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493753856:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760749.836943:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24666112, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:0.0:1789760749.836952:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ad4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.836998:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760749.837002:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.837008:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ad4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.837023:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ad4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.901607:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973341 is committed 00002000:00010000:0.0:1789760749.901936:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24666112 -> 24670208 00010000:00010000:0.0:1789760749.901952:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ad4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.901998:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ad4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.902007:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ad4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.902014:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760749.902090:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ad4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.902101:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ad4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.902120:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ad4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.902128:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ad4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.902212:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493753856:12345-192.168.203.60@tcp:4:dd.0 Request processed in 65324us (65620us total) trans 4294973341 rc 0/0 00000100:00100000:0.0:1789760749.902222:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6052 00000100:00100000:0.0:1789760749.902232:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760133490439 time 760177908519 state 0x2 ref 0 00000100:00100000:2.0:1789760749.912058:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760749.912190:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493753984 00000100:00100000:3.0:1789760749.912199:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493753984 00000100:00100000:3.0:1789760749.912255:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760133490439 time 760177908519 state 0x0 ref 1 00000100:00100000:3.0:1789760749.912274:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760178168935 time 760177908519 state 0x1 ref 1 00000100:00100000:3.0:1789760749.912309:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760178168935 time 760253344123 state 0x0 ref 1 00000100:00100000:3.0:1789760749.912321:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493753984:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760749.912372:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24670208, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:3.0:1789760749.912381:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05adb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.912397:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760749.912401:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.912407:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05adb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.912430:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05adb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760749.956345:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973342 is committed 00002000:00010000:2.0:1789760749.956515:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24670208 -> 24674304 00010000:00010000:2.0:1789760749.956530:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05adb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.956543:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05adb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.956560:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05adb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.956568:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760749.956572:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05adb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.956582:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05adb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.956607:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05adb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.956615:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05adb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.956713:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493753984:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44390us (44658us total) trans 4294973342 rc 0/0 00000100:00100000:2.0:1789760749.956724:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6053 00000100:00100000:2.0:1789760749.956734:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760178168935 time 760253344123 state 0x2 ref 0 00000100:00100000:3.0:1789760749.964246:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760749.964352:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493754112 00000100:00100000:1.0:1789760749.964362:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493754112 00000100:00100000:1.0:1789760749.964425:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760178168935 time 760253344123 state 0x0 ref 1 00000100:00100000:1.0:1789760749.964441:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760253604539 time 760253344123 state 0x1 ref 1 00000100:00100000:1.0:1789760749.964574:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760253604539 time 760305608130 state 0x0 ref 1 00000100:00100000:1.0:1789760749.964587:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493754112:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760749.964633:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24674304, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:1.0:1789760749.964641:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ae2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.964663:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760749.964668:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760749.964674:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ae2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.964688:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ae2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760750.003529:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24674304 -> 24678400 00010000:00010000:0.0:1789760750.003549:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ae2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.003567:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ae2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.003577:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ae2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.003585:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760750.003590:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ae2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.003599:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ae2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.003619:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ae2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.003628:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ae2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.003734:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493754112:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39143us (39492us total) trans 4294973343 rc 0/0 00000100:00100000:0.0:1789760750.003746:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6054 00000100:00100000:0.0:1789760750.003755:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760253604539 time 760305608130 state 0x2 ref 0 00000001:00080000:2.0:1789760750.006287:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973343 is committed 00000100:00100000:3.0:1789760750.010539:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760750.010736:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493754240 00000100:00100000:0.0:1789760750.010745:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493754240 00000100:00100000:0.0:1789760750.010804:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760253604539 time 760305608130 state 0x0 ref 1 00000100:00100000:0.0:1789760750.010819:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760305868546 time 760305608130 state 0x1 ref 1 00000100:00100000:0.0:1789760750.010851:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760305868546 time 760351886249 state 0x0 ref 1 00000100:00100000:0.0:1789760750.010863:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493754240:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760750.011132:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24678400, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:0.0:1789760750.011142:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ae9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.011157:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760750.011162:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760750.011168:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ae9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.011182:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ae9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760750.081147:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24678400 -> 24682496 00010000:00010000:0.0:1789760750.081167:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ae9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.081180:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ae9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.081193:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ae9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.081200:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760750.081204:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ae9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.081214:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ae9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.081231:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ae9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.081239:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05ae9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.081341:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493754240:12345-192.168.203.60@tcp:4:dd.0 Request processed in 70475us (70807us total) trans 4294973344 rc 0/0 00000100:00100000:0.0:1789760750.081352:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6055 00000100:00100000:0.0:1789760750.081362:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760305868546 time 760351886249 state 0x2 ref 0 00000001:00080000:1.0:1789760750.081770:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973344 is committed 00000100:00100000:3.0:1789760750.085460:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760750.085741:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493754368 00000100:00100000:0.0:1789760750.085751:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493754368 00000100:00100000:0.0:1789760750.085806:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760305868546 time 760351886249 state 0x0 ref 1 00000100:00100000:0.0:1789760750.085821:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760352146665 time 760351886249 state 0x1 ref 1 00000100:00100000:0.0:1789760750.085855:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760352146665 time 760426890765 state 0x0 ref 1 00000100:00100000:0.0:1789760750.085868:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493754368:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760750.085914:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24682496, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:0.0:1789760750.085923:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05af0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.085938:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760750.085942:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760750.085948:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05af0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.086013:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05af0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.138377:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760750.183385:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24682496 -> 24686592 00010000:00010000:2.0:1789760750.183403:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05af0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.183427:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05af0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.183436:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05af0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.183443:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760750.183447:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05af0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.183457:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05af0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.183484:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05af0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.183492:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05af0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.183588:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493754368:12345-192.168.203.60@tcp:4:dd.0 Request processed in 97718us (98133us total) trans 4294973345 rc 0/0 00000100:00100000:2.0:1789760750.183599:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6056 00000100:00100000:2.0:1789760750.183609:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760352146665 time 760426890765 state 0x2 ref 0 00000001:00080000:1.0:1789760750.184089:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973345 is committed 00000100:00100000:2.0:1789760750.195229:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760750.195375:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493754496 00000100:00100000:3.0:1789760750.195384:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493754496 00000100:00100000:3.0:1789760750.195444:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760352146665 time 760426890765 state 0x0 ref 1 00000100:00100000:3.0:1789760750.195458:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760427151181 time 760426890765 state 0x1 ref 1 00000100:00100000:3.0:1789760750.195499:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760427151181 time 760536534761 state 0x0 ref 1 00000100:00100000:3.0:1789760750.195511:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493754496:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760750.195558:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24686592, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:3.0:1789760750.195567:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05af7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.195582:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760750.195587:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.195592:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05af7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.195605:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05af7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760750.236922:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973346 is committed 00002000:00010000:3.0:1789760750.237453:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24686592 -> 24690688 00010000:00010000:3.0:1789760750.237526:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05af7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.237540:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05af7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.237550:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05af7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.237557:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760750.237562:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05af7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.237571:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05af7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.237589:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05af7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.237597:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05af7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.237748:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493754496:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42234us (42524us total) trans 4294973346 rc 0/0 00000100:00100000:3.0:1789760750.237759:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6057 00000100:00100000:3.0:1789760750.237769:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760427151181 time 760536534761 state 0x2 ref 0 00000100:00100000:0.0:1789760750.244340:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760750.244439:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493754624 00000100:00100000:1.0:1789760750.244449:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493754624 00000100:00100000:1.0:1789760750.244563:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760427151181 time 760536534761 state 0x0 ref 1 00000100:00100000:1.0:1789760750.244578:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760536795177 time 760536534761 state 0x1 ref 1 00000100:00100000:1.0:1789760750.245381:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760536795177 time 760586412282 state 0x0 ref 1 00000100:00100000:1.0:1789760750.245403:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493754624:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760750.245464:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24690688, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:1.0:1789760750.245482:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05afe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.245499:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760750.245504:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760750.245512:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05afe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.245526:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05afe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760750.285715:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973347 is committed 00002000:00010000:0.0:1789760750.286648:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24690688 -> 24694784 00010000:00010000:0.0:1789760750.286666:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05afe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.286679:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05afe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.286687:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05afe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.286701:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760750.286705:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05afe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.286714:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05afe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.286733:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05afe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.286741:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05afe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.286884:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493754624:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41483us (42549us total) trans 4294973347 rc 0/0 00000100:00100000:0.0:1789760750.286903:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6058 00000100:00100000:0.0:1789760750.286912:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760536795177 time 760586412282 state 0x2 ref 0 00000100:00100000:2.0:1789760750.294288:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760750.294378:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493754752 00000100:00100000:0.0:1789760750.294387:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493754752 00000100:00100000:0.0:1789760750.294447:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760536795177 time 760586412282 state 0x0 ref 1 00000100:00100000:0.0:1789760750.294461:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760586672698 time 760586412282 state 0x1 ref 1 00000100:00100000:0.0:1789760750.294493:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760586672698 time 760635528969 state 0x0 ref 1 00000100:00100000:0.0:1789760750.294508:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493754752:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760750.294553:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24694784, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:0.0:1789760750.294561:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b05 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.294576:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760750.294580:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760750.294586:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.294600:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760750.368339:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973348 is committed 00002000:00010000:0.0:1789760750.368470:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24694784 -> 24698880 00010000:00010000:0.0:1789760750.368486:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.368499:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b05 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.368508:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.368515:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760750.368520:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.368529:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b05 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.368548:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b05 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.368555:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b05 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.368646:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493754752:12345-192.168.203.60@tcp:4:dd.0 Request processed in 74134us (74362us total) trans 4294973348 rc 0/0 00000100:00100000:0.0:1789760750.368656:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6059 00000100:00100000:0.0:1789760750.368664:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760586672698 time 760635528969 state 0x2 ref 0 00000100:00100000:0.0:1789760750.380468:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760750.380540:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493754880 00000100:00100000:2.0:1789760750.380551:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493754880 00000100:00100000:2.0:1789760750.380605:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760586672698 time 760635528969 state 0x0 ref 1 00000100:00100000:2.0:1789760750.380621:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760635789385 time 760635528969 state 0x1 ref 1 00000100:00100000:2.0:1789760750.380653:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760635789385 time 760721688143 state 0x0 ref 1 00000100:00100000:2.0:1789760750.380665:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493754880:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760750.380712:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24698880, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:2.0:1789760750.380720:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05b0c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.380735:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760750.380740:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.380746:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05b0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.380761:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05b0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760750.412673:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973349 is committed 00002000:00010000:2.0:1789760750.412767:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24698880 -> 24702976 00010000:00010000:2.0:1789760750.412783:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05b0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.412796:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05b0c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.412806:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05b0c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.412814:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760750.412819:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05b0c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.412830:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05b0c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.412850:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05b0c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.412858:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e05b0c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.412949:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493754880:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32281us (32485us total) trans 4294973349 rc 0/0 00000100:00100000:2.0:1789760750.412999:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6060 00000100:00100000:2.0:1789760750.413009:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760635789385 time 760721688143 state 0x2 ref 0 00000100:00100000:3.0:1789760750.420124:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760750.420329:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493755008 00000100:00100000:0.0:1789760750.420339:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493755008 00000100:00100000:0.0:1789760750.420392:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760635789385 time 760721688143 state 0x0 ref 1 00000100:00100000:0.0:1789760750.420405:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760721948559 time 760721688143 state 0x1 ref 1 00000100:00100000:0.0:1789760750.420432:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760721948559 time 760761468488 state 0x0 ref 1 00000100:00100000:0.0:1789760750.420443:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493755008:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760750.420503:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24702976, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:0.0:1789760750.420512:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b13 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.420527:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760750.420532:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760750.420538:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.420553:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760750.451242:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24702976 -> 24707072 00010000:00010000:0.0:1789760750.451262:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.451275:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b13 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.451285:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b13 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.451293:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760750.451298:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b13 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.451306:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b13 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.451328:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b13 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.451335:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b13 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.451445:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493755008:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30999us (31327us total) trans 4294973350 rc 0/0 00000100:00100000:0.0:1789760750.451455:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6061 00000100:00100000:0.0:1789760750.451473:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760721948559 time 760761468488 state 0x2 ref 0 00000001:00080000:2.0:1789760750.453344:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973350 is committed 00000100:00100000:3.0:1789760750.455925:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760750.456395:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493755136 00000100:00100000:0.0:1789760750.456422:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493755136 00000100:00100000:0.0:1789760750.456602:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760721948559 time 760761468488 state 0x0 ref 1 00000100:00100000:0.0:1789760750.456618:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760761728904 time 760761468488 state 0x1 ref 1 00000100:00100000:0.0:1789760750.456689:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760761728904 time 760797724244 state 0x0 ref 1 00000100:00100000:0.0:1789760750.456702:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493755136:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760750.456752:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24707072, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:0.0:1789760750.456761:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05b1a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.456777:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760750.456782:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760750.456788:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05b1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.456811:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05b1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.494385:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973351 is committed 00002000:00010000:0.0:1789760750.494538:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24707072 -> 24711168 00010000:00010000:0.0:1789760750.494555:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05b1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.494568:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05b1a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.494577:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05b1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.494585:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760750.494590:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05b1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.494600:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05b1a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.494619:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05b1a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.494626:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05b1a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.494747:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493755136:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38043us (38828us total) trans 4294973351 rc 0/0 00000100:00100000:0.0:1789760750.494757:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6062 00000100:00100000:0.0:1789760750.494767:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760761728904 time 760797724244 state 0x2 ref 0 00000100:00100000:0.0:1789760750.499681:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760750.500125:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493755264 00000100:00100000:3.0:1789760750.500135:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493755264 00000100:00100000:2.0:1789760750.500263:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760761728904 time 760797724244 state 0x0 ref 1 00000100:00100000:2.0:1789760750.500279:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760797984660 time 760797724244 state 0x1 ref 1 00000100:00100000:2.0:1789760750.500333:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760797984660 time 760841368961 state 0x0 ref 1 00000100:00100000:2.0:1789760750.500346:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493755264:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760750.500392:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24711168, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:2.0:1789760750.500400:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05b21 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.500426:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760750.500431:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.500438:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05b21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.500452:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05b21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760750.538516:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24711168 -> 24715264 00010000:00010000:0.0:1789760750.538536:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05b21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.538548:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05b21 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.538556:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05b21 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.538563:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760750.538567:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05b21 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.538575:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05b21 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.538596:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05b21 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.538608:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05b21 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.538718:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493755264:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38368us (39040us total) trans 4294973352 rc 0/0 00000100:00100000:0.0:1789760750.538729:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6063 00000100:00100000:0.0:1789760750.538739:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760797984660 time 760841368961 state 0x2 ref 0 00000001:00080000:3.0:1789760750.540656:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973352 is committed 00000100:00100000:0.0:1789760750.543804:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760750.543908:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493755392 00000100:00100000:2.0:1789760750.544047:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493755392 00000100:00100000:2.0:1789760750.544119:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760797984660 time 760841368961 state 0x0 ref 1 00000100:00100000:2.0:1789760750.544136:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760841629377 time 760841368961 state 0x1 ref 1 00000100:00100000:2.0:1789760750.544155:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760841629377 time 760885190941 state 0x0 ref 1 00000100:00100000:2.0:1789760750.544168:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493755392:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760750.544217:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24715264, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:2.0:1789760750.544226:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b28 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.544241:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760750.544245:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.544252:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.544266:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760750.572237:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973353 is committed 00002000:00010000:2.0:1789760750.572322:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24715264 -> 24719360 00010000:00010000:2.0:1789760750.572337:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.572350:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b28 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.572359:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b28 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.572368:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760750.572373:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b28 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.572382:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b28 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.572401:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b28 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.572408:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b28 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.572583:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493755392:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28412us (28783us total) trans 4294973353 rc 0/0 00000100:00100000:2.0:1789760750.572599:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6064 00000100:00100000:2.0:1789760750.572608:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760841629377 time 760885190941 state 0x2 ref 0 00000100:00100000:1.0:1789760750.577793:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760750.578090:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493755520 00000100:00100000:0.0:1789760750.578099:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493755520 00000100:00100000:0.0:1789760750.578153:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760841629377 time 760885190941 state 0x0 ref 1 00000100:00100000:0.0:1789760750.578168:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760885451357 time 760885190941 state 0x1 ref 1 00000100:00100000:0.0:1789760750.578279:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760885451357 time 760919313216 state 0x0 ref 1 00000100:00100000:0.0:1789760750.578293:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493755520:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760750.578344:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24719360, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xc007 00010000:00010000:0.0:1789760750.578352:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b2f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.578367:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760750.578371:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760750.578378:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.578391:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760750.610588:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973354 is committed 00002000:00010000:3.0:1789760750.610703:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24719360 -> 24723456 00002000:00010000:3.0:1789760750.610708:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760749 -> 1789760750 00002000:00010000:3.0:1789760750.610710:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760749 -> 1789760750 00010000:00010000:3.0:1789760750.610721:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.610737:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b2f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.610746:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.610752:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760750.610757:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.610765:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b2f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.610785:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b2f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.610793:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e05b2f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.611112:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493755520:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32816us (33323us total) trans 4294973354 rc 0/0 00000100:00100000:3.0:1789760750.611122:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6065 00000100:00100000:3.0:1789760750.611131:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760885451357 time 760919313216 state 0x2 ref 0 00000100:00100000:0.0:1789760750.614869:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760750.615195:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493755648 00000100:00100000:1.0:1789760750.615204:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493755648 00000100:00100000:1.0:1789760750.615259:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760885451357 time 760919313216 state 0x0 ref 1 00000100:00100000:1.0:1789760750.615274:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760919573632 time 760919313216 state 0x1 ref 1 00000100:00100000:1.0:1789760750.615342:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760919573632 time 760956377071 state 0x0 ref 1 00000100:00100000:1.0:1789760750.615355:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493755648:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760750.615423:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24723456, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:1.0:1789760750.615432:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05b36 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.615447:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760750.615452:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760750.615459:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05b36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.615479:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05b36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760750.645904:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973355 is committed 00002000:00010000:3.0:1789760750.646159:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24723456 -> 24727552 00010000:00010000:3.0:1789760750.646176:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05b36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.646188:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05b36 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.646197:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05b36 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.646205:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760750.646210:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05b36 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.646218:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05b36 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.646237:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05b36 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.646244:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05b36 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.646337:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493755648:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30979us (31473us total) trans 4294973355 rc 0/0 00000100:00100000:3.0:1789760750.646348:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6066 00000100:00100000:3.0:1789760750.646358:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760919573632 time 760956377071 state 0x2 ref 0 00000100:00100000:0.0:1789760750.652327:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760750.652405:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493755776 00000100:00100000:1.0:1789760750.652423:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493755776 00000100:00100000:1.0:1789760750.652489:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760919573632 time 760956377071 state 0x0 ref 1 00000100:00100000:1.0:1789760750.652503:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760956637487 time 760956377071 state 0x1 ref 1 00000100:00100000:1.0:1789760750.652530:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760956637487 time 760993566066 state 0x0 ref 1 00000100:00100000:1.0:1789760750.652541:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493755776:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760750.652590:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24727552, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:1.0:1789760750.652600:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05b3d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.652614:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760750.652618:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760750.652624:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05b3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.652638:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05b3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760750.743937:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973356 is committed 00002000:00010000:3.0:1789760750.744210:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24727552 -> 24731648 00010000:00010000:3.0:1789760750.744226:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05b3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.744239:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05b3d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.744248:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05b3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.744254:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760750.744259:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05b3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.744267:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05b3d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.744291:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05b3d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.744298:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05b3d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.744386:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493755776:12345-192.168.203.60@tcp:4:dd.0 Request processed in 91841us (92064us total) trans 4294973356 rc 0/0 00000100:00100000:3.0:1789760750.744396:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6067 00000100:00100000:3.0:1789760750.744406:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760956637487 time 760993566066 state 0x2 ref 0 00000100:00100000:0.0:1789760750.752865:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760750.752995:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493755904 00000100:00100000:2.0:1789760750.753005:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493755904 00000100:00100000:2.0:1789760750.753056:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760956637487 time 760993566066 state 0x0 ref 1 00000100:00100000:2.0:1789760750.753072:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760993826482 time 760993566066 state 0x1 ref 1 00000100:00100000:2.0:1789760750.753096:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760993826482 time 761094131585 state 0x0 ref 1 00000100:00100000:2.0:1789760750.753106:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb07850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493755904:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760750.753153:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24731648, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:2.0:1789760750.753162:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b44 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.753178:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760750.753182:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.753189:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.753203:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760750.797550:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973357 is committed 00002000:00010000:2.0:1789760750.797657:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24731648 -> 24735744 00010000:00010000:2.0:1789760750.797674:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.797687:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b44 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.797696:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b44 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.797703:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760750.797708:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b44 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.797717:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b44 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.797736:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b44 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.797743:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b44 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.797873:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb07850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493755904:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44762us (45012us total) trans 4294973357 rc 0/0 00000100:00100000:2.0:1789760750.797884:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6068 00000100:00100000:2.0:1789760750.797894:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760993826482 time 761094131585 state 0x2 ref 0 00000100:00100000:0.0:1789760750.809923:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760750.810059:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493756032 00000100:00100000:3.0:1789760750.810068:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493756032 00000100:00100000:3.0:1789760750.810117:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 760993826482 time 761094131585 state 0x0 ref 1 00000100:00100000:3.0:1789760750.810133:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761094392001 time 761094131585 state 0x1 ref 1 00000100:00100000:3.0:1789760750.810164:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761094392001 time 761151199969 state 0x0 ref 1 00000100:00100000:3.0:1789760750.810176:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493756032:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760750.810227:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24735744, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:3.0:1789760750.810235:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05b4b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.810255:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760750.810259:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.810265:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05b4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.810277:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05b4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760750.860132:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24735744 -> 24739840 00010000:00010000:0.0:1789760750.860151:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05b4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.860164:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05b4b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.860173:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05b4b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.860191:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760750.860196:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05b4b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.860206:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05b4b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.860225:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05b4b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.860234:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05b4b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.860320:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493756032:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50140us (50401us total) trans 4294973358 rc 0/0 00000100:00100000:0.0:1789760750.860330:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6069 00000100:00100000:0.0:1789760750.860340:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761094392001 time 761151199969 state 0x2 ref 0 00000001:00080000:2.0:1789760750.860937:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973358 is committed 00000100:00100000:0.0:1789760750.865083:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760750.868002:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493756160 00000100:00100000:3.0:1789760750.868015:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493756160 00000100:00100000:3.0:1789760750.868075:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761094392001 time 761151199969 state 0x0 ref 1 00000100:00100000:3.0:1789760750.868090:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761151460385 time 761151199969 state 0x1 ref 1 00000100:00100000:3.0:1789760750.868124:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761151460385 time 761209159284 state 0x0 ref 1 00000100:00100000:3.0:1789760750.868136:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493756160:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760750.868184:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24739840, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:3.0:1789760750.868193:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05b52 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.868208:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760750.868212:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.868219:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05b52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.868233:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05b52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760750.912404:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973359 is committed 00002000:00010000:3.0:1789760750.912594:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24739840 -> 24743936 00010000:00010000:3.0:1789760750.912613:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05b52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.912625:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05b52 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.912633:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05b52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.912641:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760750.912645:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05b52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.912655:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05b52 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.912675:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05b52 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.912683:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05b52 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.912768:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493756160:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44629us (47689us total) trans 4294973359 rc 0/0 00000100:00100000:3.0:1789760750.912778:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6070 00000100:00100000:3.0:1789760750.912788:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761151460385 time 761209159284 state 0x2 ref 0 00000100:00100000:3.0:1789760750.919457:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760750.920068:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493756288 00000100:00100000:1.0:1789760750.920080:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493756288 00000100:00100000:1.0:1789760750.920140:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761151460385 time 761209159284 state 0x0 ref 1 00000100:00100000:1.0:1789760750.920159:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761209419700 time 761209159284 state 0x1 ref 1 00000100:00100000:1.0:1789760750.920197:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761209419700 time 761261232210 state 0x0 ref 1 00000100:00100000:1.0:1789760750.920209:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493756288:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760750.920260:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24743936, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:1.0:1789760750.920269:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05b59 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.920284:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760750.920289:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760750.920295:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05b59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.920308:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05b59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760750.967506:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24743936 -> 24748032 00010000:00010000:0.0:1789760750.967524:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05b59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.967537:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05b59 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.967546:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05b59 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.967553:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760750.967558:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05b59 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.967572:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05b59 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.967592:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05b59 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.967600:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05b59 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.967700:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493756288:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47487us (48246us total) trans 4294973360 rc 0/0 00000100:00100000:0.0:1789760750.967710:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6071 00000100:00100000:0.0:1789760750.967720:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761209419700 time 761261232210 state 0x2 ref 0 00000001:00080000:1.0:1789760750.968029:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973360 is committed 00000100:00100000:3.0:1789760750.976668:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760750.977513:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493756416 00000100:00100000:0.0:1789760750.977523:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493756416 00000100:00100000:0.0:1789760750.977580:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761209419700 time 761261232210 state 0x0 ref 1 00000100:00100000:0.0:1789760750.977596:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761261492626 time 761261232210 state 0x1 ref 1 00000100:00100000:0.0:1789760750.977632:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761261492626 time 761318667680 state 0x0 ref 1 00000100:00100000:0.0:1789760750.977646:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493756416:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760750.977697:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24748032, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:0.0:1789760750.977705:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05b60 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.977721:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760750.977725:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760750.977731:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05b60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.977746:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05b60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760751.033270:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973361 is committed 00002000:00010000:1.0:1789760751.033474:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24748032 -> 24752128 00010000:00010000:1.0:1789760751.033490:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05b60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.033501:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05b60 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.033509:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05b60 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.033518:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760751.033523:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05b60 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.033644:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05b60 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.033666:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05b60 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.033674:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05b60 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.033830:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493756416:12345-192.168.203.60@tcp:4:dd.0 Request processed in 56182us (57167us total) trans 4294973361 rc 0/0 00000100:00100000:1.0:1789760751.033841:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6072 00000100:00100000:1.0:1789760751.033851:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761261492626 time 761318667680 state 0x2 ref 0 00000100:00100000:3.0:1789760751.042371:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760751.042593:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493756544 00000100:00100000:0.0:1789760751.042603:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493756544 00000100:00100000:0.0:1789760751.042677:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761261492626 time 761318667680 state 0x0 ref 1 00000100:00100000:0.0:1789760751.042692:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761318928096 time 761318667680 state 0x1 ref 1 00000100:00100000:0.0:1789760751.042723:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761318928096 time 761383758946 state 0x0 ref 1 00000100:00100000:0.0:1789760751.042737:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493756544:12345-192.168.203.60@tcp:4:dd.0 00000100:00080000:1.0:1789760751.042834:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789760751.042893:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24752128, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:0.0:1789760751.042901:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05b67 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.042916:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760751.042927:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760751.042934:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05b67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.042953:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05b67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760751.101648:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973362 is committed 00002000:00010000:0.0:1789760751.102136:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24752128 -> 24756224 00010000:00010000:0.0:1789760751.102152:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05b67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.102164:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05b67 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.102172:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05b67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.102180:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760751.102185:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05b67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.102193:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05b67 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.102212:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05b67 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.102220:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05b67 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.102357:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493756544:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59618us (59990us total) trans 4294973362 rc 0/0 00000100:00100000:0.0:1789760751.102368:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6073 00000100:00100000:0.0:1789760751.102377:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761318928096 time 761383758946 state 0x2 ref 0 00000100:00100000:3.0:1789760751.105861:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760751.106306:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493756672 00000100:00100000:0.0:1789760751.106316:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493756672 00000100:00100000:0.0:1789760751.106434:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761318928096 time 761383758946 state 0x0 ref 1 00000100:00100000:0.0:1789760751.106449:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761384019362 time 761383758946 state 0x1 ref 1 00000100:00100000:0.0:1789760751.106513:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761384019362 time 761447548625 state 0x0 ref 1 00000100:00100000:0.0:1789760751.106526:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8555180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493756672:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760751.106576:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24756224, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:0.0:1789760751.106585:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05b6e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.106600:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760751.106605:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760751.106610:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05b6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.106624:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05b6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760751.155944:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973363 is committed 00002000:00010000:0.0:1789760751.156212:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24756224 -> 24760320 00010000:00010000:0.0:1789760751.156229:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05b6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.156240:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05b6e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.156248:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05b6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.156255:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760751.156260:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05b6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.156268:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05b6e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.156288:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05b6e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.156295:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05b6e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.156424:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8555180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493756672:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49894us (50567us total) trans 4294973363 rc 0/0 00000100:00100000:0.0:1789760751.156438:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6074 00000100:00100000:0.0:1789760751.156448:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761384019362 time 761447548625 state 0x2 ref 0 00010000:00010000:0.0:1789760751.162079:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760751.166454:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760751.166569:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493756800 00000100:00100000:2.0:1789760751.166578:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493756800 00000100:00100000:2.0:1789760751.166636:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761384019362 time 761447548625 state 0x0 ref 1 00000100:00100000:2.0:1789760751.166651:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761447809041 time 761447548625 state 0x1 ref 1 00000100:00100000:2.0:1789760751.166713:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761447809041 time 761507747708 state 0x0 ref 1 00000100:00100000:2.0:1789760751.166737:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493756800:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760751.166785:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24760320, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:2.0:1789760751.166794:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b75 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.166821:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760751.166825:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760751.166831:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.166843:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760751.203902:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973364 is committed 00002000:00010000:0.0:1789760751.204063:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24760320 -> 24764416 00010000:00010000:0.0:1789760751.204080:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.204093:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b75 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.204102:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.204110:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760751.204115:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.204124:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b75 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.204143:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b75 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.204151:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e05b75 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.204278:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493756800:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37548us (37828us total) trans 4294973364 rc 0/0 00000100:00100000:0.0:1789760751.204289:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6075 00000100:00100000:0.0:1789760751.204299:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761447809041 time 761507747708 state 0x2 ref 0 00000100:00100000:3.0:1789760751.207863:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760751.208098:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493756928 00000100:00100000:1.0:1789760751.208109:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493756928 00000100:00100000:1.0:1789760751.208173:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761447809041 time 761507747708 state 0x0 ref 1 00000100:00100000:1.0:1789760751.208189:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761508008124 time 761507747708 state 0x1 ref 1 00000100:00100000:1.0:1789760751.208252:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761508008124 time 761549287478 state 0x0 ref 1 00000100:00100000:1.0:1789760751.208267:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493756928:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760751.208315:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24764416, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:1.0:1789760751.208325:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05b7c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.208341:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760751.208345:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.208352:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05b7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.208366:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05b7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760751.251037:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24764416 -> 24768512 00010000:00010000:1.0:1789760751.251057:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05b7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.251069:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05b7c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.251079:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05b7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.251086:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760751.251092:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05b7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.251101:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05b7c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.251122:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05b7c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.251130:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05b7c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.251221:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493756928:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42951us (43367us total) trans 4294973365 rc 0/0 00000100:00100000:1.0:1789760751.251231:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6076 00000100:00100000:1.0:1789760751.251241:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761508008124 time 761549287478 state 0x2 ref 0 00000001:00080000:3.0:1789760751.252241:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973365 is committed 00000100:00100000:0.0:1789760751.256850:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760751.257696:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493757056 00000100:00100000:2.0:1789760751.257708:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493757056 00000100:00100000:2.0:1789760751.257768:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761508008124 time 761549287478 state 0x0 ref 1 00000100:00100000:2.0:1789760751.257784:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761549547894 time 761549287478 state 0x1 ref 1 00000100:00100000:2.0:1789760751.257854:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761549547894 time 761598889385 state 0x0 ref 1 00000100:00100000:2.0:1789760751.257868:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493757056:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760751.257924:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24768512, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:2.0:1789760751.257932:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05b83 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.257948:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760751.257952:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760751.257958:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05b83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.257985:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05b83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760751.299574:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973366 is committed 00002000:00010000:2.0:1789760751.300578:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24768512 -> 24772608 00010000:00010000:2.0:1789760751.300595:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05b83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.300608:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05b83 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.300618:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05b83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.300626:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760751.300631:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05b83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.300641:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05b83 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.300660:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05b83 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.300668:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05b83 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760751.301232:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493757056:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43361us (44389us total) trans 4294973366 rc 0/0 00000100:00100000:2.0:1789760751.301244:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6077 00000100:00100000:2.0:1789760751.301254:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761549547894 time 761598889385 state 0x2 ref 0 00000100:00100000:0.0:1789760751.304139:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760751.304202:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493757184 00000100:00100000:3.0:1789760751.304212:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493757184 00000100:00100000:3.0:1789760751.304263:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761549547894 time 761598889385 state 0x0 ref 1 00000100:00100000:3.0:1789760751.304277:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761599149801 time 761598889385 state 0x1 ref 1 00000100:00100000:3.0:1789760751.304309:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761599149801 time 761645344546 state 0x0 ref 1 00000100:00100000:3.0:1789760751.304321:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493757184:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760751.304370:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24772608, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:3.0:1789760751.304379:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05b8a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.304394:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760751.304399:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.304421:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05b8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.304437:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05b8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760751.355856:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973367 is committed 00002000:00010000:1.0:1789760751.356233:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24772608 -> 24776704 00010000:00010000:1.0:1789760751.356249:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05b8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.356262:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05b8a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.356272:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05b8a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.356284:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760751.356289:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05b8a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.356297:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05b8a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.356316:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05b8a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.356324:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05b8a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.356475:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493757184:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52151us (52342us total) trans 4294973367 rc 0/0 00000100:00100000:1.0:1789760751.356485:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6078 00000100:00100000:1.0:1789760751.356495:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761599149801 time 761645344546 state 0x2 ref 0 00000100:00100000:3.0:1789760751.360736:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760751.361307:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493757312 00000100:00100000:0.0:1789760751.361318:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493757312 00000100:00100000:0.0:1789760751.361378:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761599149801 time 761645344546 state 0x0 ref 1 00000100:00100000:0.0:1789760751.361395:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761645604962 time 761645344546 state 0x1 ref 1 00000100:00100000:0.0:1789760751.361465:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761645604962 time 761702500104 state 0x0 ref 1 00000100:00100000:0.0:1789760751.361479:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8556a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493757312:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760751.361530:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24776704, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:0.0:1789760751.361541:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05b91 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.361561:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:0.0:1789760751.361566:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760751.361573:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05b91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.361591:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05b91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760751.428798:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24776704 -> 24780800 00010000:00010000:1.0:1789760751.428819:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05b91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.428831:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05b91 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.428845:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05b91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.428852:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760751.428856:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05b91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.428866:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05b91 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.428884:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05b91 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.428892:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05b91 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.429013:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8556a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493757312:12345-192.168.203.60@tcp:4:dd.0 Request processed in 67532us (68286us total) trans 4294973368 rc 0/0 00000100:00100000:1.0:1789760751.429026:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6079 00000100:00100000:1.0:1789760751.429034:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761645604962 time 761702500104 state 0x2 ref 0 00000001:00080000:3.0:1789760751.429492:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973368 is committed 00000100:00100000:0.0:1789760751.435640:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760751.435760:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493757440 00000100:00100000:1.0:1789760751.435770:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493757440 00000100:00100000:1.0:1789760751.436377:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761645604962 time 761702500104 state 0x0 ref 1 00000100:00100000:1.0:1789760751.436393:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761702760520 time 761702500104 state 0x1 ref 1 00000100:00100000:1.0:1789760751.436484:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761702760520 time 761777519427 state 0x0 ref 1 00000100:00100000:1.0:1789760751.436498:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493757440:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760751.436544:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24780800, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:1.0:1789760751.436552:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05b98 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.436567:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760751.436571:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.436577:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05b98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.436591:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05b98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760751.483982:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24780800 -> 24784896 00010000:00010000:2.0:1789760751.484002:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05b98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.484014:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05b98 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.484022:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05b98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.484034:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760751.484039:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05b98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.484047:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05b98 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.484065:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05b98 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.484073:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05b98 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760751.484161:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493757440:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47661us (48526us total) trans 4294973369 rc 0/0 00000100:00100000:2.0:1789760751.484172:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6080 00000100:00100000:2.0:1789760751.484182:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761702760520 time 761777519427 state 0x2 ref 0 00000001:00080000:3.0:1789760751.484431:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973369 is committed 00000100:00100000:1.0:1789760751.488522:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760751.488615:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493757568 00000100:00100000:2.0:1789760751.488624:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493757568 00000100:00100000:2.0:1789760751.488682:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761702760520 time 761777519427 state 0x0 ref 1 00000100:00100000:2.0:1789760751.488697:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761777779843 time 761777519427 state 0x1 ref 1 00000100:00100000:2.0:1789760751.488727:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761777779843 time 761829763035 state 0x0 ref 1 00000100:00100000:2.0:1789760751.488744:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493757568:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760751.488795:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24784896, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:2.0:1789760751.488805:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05b9f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.488822:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760751.488827:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760751.488833:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05b9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.488846:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05b9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760751.533849:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973370 is committed 00002000:00010000:0.0:1789760751.536606:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24784896 -> 24788992 00010000:00010000:0.0:1789760751.536624:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05b9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.536637:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05b9f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.536646:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05b9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.536653:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760751.536658:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05b9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.536667:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05b9f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.536685:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05b9f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.536693:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e05b9f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.536848:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493757568:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48101us (48330us total) trans 4294973370 rc 0/0 00000100:00100000:0.0:1789760751.536858:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6081 00000100:00100000:0.0:1789760751.536867:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761777779843 time 761829763035 state 0x2 ref 0 00000100:00100000:1.0:1789760751.542544:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760751.542718:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493757696 00000100:00100000:3.0:1789760751.542728:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493757696 00000100:00100000:3.0:1789760751.542782:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761777779843 time 761829763035 state 0x0 ref 1 00000100:00100000:3.0:1789760751.542798:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761830023451 time 761829763035 state 0x1 ref 1 00000100:00100000:3.0:1789760751.542832:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761830023451 time 761883867622 state 0x0 ref 1 00000100:00100000:3.0:1789760751.542844:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493757696:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760751.543357:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24788992, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:3.0:1789760751.543364:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05ba6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.543377:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760751.543380:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.543385:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05ba6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.543415:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05ba6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760751.589841:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973371 is committed 00002000:00010000:3.0:1789760751.589943:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24788992 -> 24793088 00010000:00010000:3.0:1789760751.589987:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05ba6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.590000:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05ba6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.590009:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05ba6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.590015:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760751.590021:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05ba6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.590030:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05ba6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.590049:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05ba6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.590057:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05ba6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.590142:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493757696:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47294us (47602us total) trans 4294973371 rc 0/0 00000100:00100000:3.0:1789760751.590152:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6082 00000100:00100000:3.0:1789760751.590161:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761830023451 time 761883867622 state 0x2 ref 0 00000100:00100000:3.0:1789760751.603870:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760751.603996:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493757824 00000100:00100000:1.0:1789760751.604006:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493757824 00000100:00100000:1.0:1789760751.604062:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761830023451 time 761883867622 state 0x0 ref 1 00000100:00100000:1.0:1789760751.604076:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761884128038 time 761883867622 state 0x1 ref 1 00000100:00100000:1.0:1789760751.604107:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761884128038 time 761945142950 state 0x0 ref 1 00000100:00100000:1.0:1789760751.604123:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493757824:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760751.604170:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24793088, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xc007 00010000:00010000:1.0:1789760751.604179:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05bad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.604195:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760751.604199:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.604206:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05bad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.604218:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05bad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760751.686668:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973372 is committed 00002000:00010000:3.0:1789760751.686768:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24793088 -> 24797184 00002000:00010000:3.0:1789760751.686773:0:13038:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760750 -> 1789760751 00002000:00010000:3.0:1789760751.686775:0:13038:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760750 -> 1789760751 00010000:00010000:3.0:1789760751.686787:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05bad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.686799:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05bad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.686814:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05bad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.686821:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760751.686826:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05bad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.686836:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05bad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.686857:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05bad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.686865:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05bad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.686986:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493757824:12345-192.168.203.60@tcp:4:dd.0 Request processed in 82834us (83095us total) trans 4294973372 rc 0/0 00000100:00100000:3.0:1789760751.686996:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6083 00000100:00100000:3.0:1789760751.687006:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761884128038 time 761945142950 state 0x2 ref 0 00000100:00100000:1.0:1789760751.692445:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760751.692575:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493757952 00000100:00100000:2.0:1789760751.692585:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493757952 00000100:00100000:2.0:1789760751.692640:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761884128038 time 761945142950 state 0x0 ref 1 00000100:00100000:2.0:1789760751.692656:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761945403366 time 761945142950 state 0x1 ref 1 00000100:00100000:2.0:1789760751.692681:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761945403366 time 762033716962 state 0x0 ref 1 00000100:00100000:2.0:1789760751.692694:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493757952:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760751.692813:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24797184, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:2.0:1789760751.692822:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05bb4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.692838:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760751.692843:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760751.692849:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05bb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.692862:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05bb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760751.759509:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24797184 -> 24801280 00010000:00010000:3.0:1789760751.759529:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05bb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.759549:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05bb4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.759557:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05bb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.759564:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760751.759570:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05bb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.759579:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05bb4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.759607:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05bb4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.759614:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05bb4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.759707:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493757952:12345-192.168.203.60@tcp:4:dd.0 Request processed in 67011us (67267us total) trans 4294973373 rc 0/0 00000100:00100000:3.0:1789760751.759717:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6084 00000100:00100000:3.0:1789760751.759728:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761945403366 time 762033716962 state 0x2 ref 0 00000001:00080000:1.0:1789760751.760284:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973373 is committed 00000100:00100000:1.0:1789760751.763216:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760751.763315:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493758080 00000100:00100000:0.0:1789760751.763324:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493758080 00000100:00100000:0.0:1789760751.763376:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 761945403366 time 762033716962 state 0x0 ref 1 00000100:00100000:0.0:1789760751.763391:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762033977378 time 762033716962 state 0x1 ref 1 00000100:00100000:0.0:1789760751.763427:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762033977378 time 762104462179 state 0x0 ref 1 00000100:00100000:0.0:1789760751.763439:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493758080:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760751.763574:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24801280, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:0.0:1789760751.763583:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05bbb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.763599:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760751.763603:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760751.763609:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05bbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.763621:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05bbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760751.812791:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973374 is committed 00002000:00010000:2.0:1789760751.812993:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24801280 -> 24805376 00010000:00010000:2.0:1789760751.813009:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05bbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.813021:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05bbb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.813029:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05bbb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.813055:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760751.813060:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05bbb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.813068:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05bbb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.813092:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05bbb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.813101:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05bbb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760751.813181:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493758080:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49740us (49969us total) trans 4294973374 rc 0/0 00000100:00100000:2.0:1789760751.813191:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6085 00000100:00100000:2.0:1789760751.813201:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762033977378 time 762104462179 state 0x2 ref 0 00000100:00100000:1.0:1789760751.816560:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760751.816677:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493758208 00000100:00100000:3.0:1789760751.816688:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493758208 00000100:00100000:3.0:1789760751.816736:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762033977378 time 762104462179 state 0x0 ref 1 00000100:00100000:3.0:1789760751.816751:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762104722595 time 762104462179 state 0x1 ref 1 00000100:00100000:3.0:1789760751.816783:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762104722595 time 762157818518 state 0x0 ref 1 00000100:00100000:3.0:1789760751.816796:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493758208:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760751.816845:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24805376, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:3.0:1789760751.816855:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05bc2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.816870:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760751.816874:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.816891:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05bc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.816907:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05bc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760751.845741:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973375 is committed 00002000:00010000:3.0:1789760751.845944:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24805376 -> 24809472 00010000:00010000:3.0:1789760751.845978:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05bc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.845993:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05bc2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.846007:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05bc2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.846015:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760751.846019:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05bc2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.846027:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05bc2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.846046:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05bc2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.846053:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05bc2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.847094:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493758208:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30292us (30535us total) trans 4294973375 rc 0/0 00000100:00100000:3.0:1789760751.847106:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6086 00000100:00100000:3.0:1789760751.847115:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762104722595 time 762157818518 state 0x2 ref 0 00000100:00100000:1.0:1789760751.850767:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760751.850943:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493758336 00000100:00100000:3.0:1789760751.850952:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493758336 00000100:00100000:3.0:1789760751.851077:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762104722595 time 762157818518 state 0x0 ref 1 00000100:00100000:3.0:1789760751.851092:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762158078934 time 762157818518 state 0x1 ref 1 00000100:00100000:3.0:1789760751.851125:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762158078934 time 762192160508 state 0x0 ref 1 00000100:00100000:3.0:1789760751.851137:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493758336:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760751.851181:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24809472, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:3.0:1789760751.851189:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05bc9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.851204:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760751.851209:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.851215:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05bc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.851229:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05bc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760751.889174:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973376 is committed 00002000:00010000:0.0:1789760751.889277:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24809472 -> 24813568 00010000:00010000:0.0:1789760751.889293:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05bc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.889305:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05bc9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.889313:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05bc9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.889319:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760751.889323:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05bc9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.889331:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05bc9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.889349:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05bc9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.889356:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05bc9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.889555:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493758336:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38414us (38886us total) trans 4294973376 rc 0/0 00000100:00100000:0.0:1789760751.889567:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6087 00000100:00100000:0.0:1789760751.889576:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762158078934 time 762192160508 state 0x2 ref 0 00000100:00100000:1.0:1789760751.893776:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760751.893871:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493758464 00000100:00100000:3.0:1789760751.893985:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493758464 00000100:00100000:3.0:1789760751.894042:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762158078934 time 762192160508 state 0x0 ref 1 00000100:00100000:3.0:1789760751.894056:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762192420924 time 762192160508 state 0x1 ref 1 00000100:00100000:3.0:1789760751.894085:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762192420924 time 762235120565 state 0x0 ref 1 00000100:00100000:3.0:1789760751.894097:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493758464:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760751.894152:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24813568, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:3.0:1789760751.894161:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05bd0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.894178:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760751.894182:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.894188:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05bd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.894202:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05bd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760751.968737:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973377 is committed 00002000:00010000:1.0:1789760751.968846:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24813568 -> 24817664 00010000:00010000:1.0:1789760751.968860:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05bd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.968872:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05bd0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.968886:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05bd0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.968894:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760751.968898:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05bd0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.968906:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05bd0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.968925:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05bd0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.968932:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05bd0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.969061:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493758464:12345-192.168.203.60@tcp:4:dd.0 Request processed in 74960us (75288us total) trans 4294973377 rc 0/0 00000100:00100000:1.0:1789760751.969071:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6088 00000100:00100000:1.0:1789760751.969082:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762192420924 time 762235120565 state 0x2 ref 0 00000100:00100000:1.0:1789760751.972345:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760751.972552:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493758592 00000100:00100000:2.0:1789760751.972562:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493758592 00000100:00100000:2.0:1789760751.972614:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762192420924 time 762235120565 state 0x0 ref 1 00000100:00100000:2.0:1789760751.972629:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762235380981 time 762235120565 state 0x1 ref 1 00000100:00100000:2.0:1789760751.972710:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762235380981 time 762313744557 state 0x0 ref 1 00000100:00100000:2.0:1789760751.972724:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493758592:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760751.972773:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24817664, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:2.0:1789760751.972782:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05bd7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.972860:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760751.972865:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760751.972930:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05bd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.972953:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05bd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760752.015531:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973378 is committed 00002000:00010000:1.0:1789760752.018705:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24817664 -> 24821760 00010000:00010000:1.0:1789760752.018725:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05bd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.018737:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05bd7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.018746:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05bd7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.018753:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760752.018758:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05bd7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.018767:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05bd7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.018784:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05bd7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.018790:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05bd7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.018932:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493758592:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46206us (46591us total) trans 4294973378 rc 0/0 00000100:00100000:1.0:1789760752.018944:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6089 00000100:00100000:1.0:1789760752.018953:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762235380981 time 762313744557 state 0x2 ref 0 00000100:00100000:3.0:1789760752.023409:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760752.023489:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493758720 00000100:00100000:0.0:1789760752.023498:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493758720 00000100:00100000:0.0:1789760752.023561:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762235380981 time 762313744557 state 0x0 ref 1 00000100:00100000:0.0:1789760752.023575:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762314004973 time 762313744557 state 0x1 ref 1 00000100:00100000:0.0:1789760752.023606:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762314004973 time 762364641334 state 0x0 ref 1 00000100:00100000:0.0:1789760752.023618:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493758720:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760752.023666:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24821760, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:0.0:1789760752.023675:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e05bde lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.023690:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760752.023694:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.023699:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e05bde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.023712:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e05bde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760752.065531:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973379 is committed 00002000:00010000:3.0:1789760752.065832:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24821760 -> 24825856 00010000:00010000:3.0:1789760752.065848:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e05bde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.065860:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e05bde lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.065869:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e05bde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.065878:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760752.065883:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e05bde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.065897:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e05bde lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.065916:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e05bde lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.065924:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e05bde lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.066058:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493758720:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42437us (42664us total) trans 4294973379 rc 0/0 00000100:00100000:3.0:1789760752.066067:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6090 00000100:00100000:3.0:1789760752.066078:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762314004973 time 762364641334 state 0x2 ref 0 00000100:00100000:3.0:1789760752.070242:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760752.070516:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493758848 00000100:00100000:0.0:1789760752.070525:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493758848 00000100:00100000:0.0:1789760752.070581:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762314004973 time 762364641334 state 0x0 ref 1 00000100:00100000:0.0:1789760752.070596:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762364901750 time 762364641334 state 0x1 ref 1 00000100:00100000:0.0:1789760752.070628:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762364901750 time 762411664154 state 0x0 ref 1 00000100:00100000:0.0:1789760752.070640:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493758848:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760752.070689:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24825856, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:0.0:1789760752.070697:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05be5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.070711:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760752.070716:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.070722:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05be5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.070735:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05be5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760752.119395:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24825856 -> 24829952 00010000:00010000:0.0:1789760752.119423:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05be5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.119437:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05be5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.119457:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05be5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.119466:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760752.119471:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05be5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.119479:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05be5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.119497:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05be5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.119505:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05be5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.119599:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493758848:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48955us (49361us total) trans 4294973380 rc 0/0 00000100:00100000:0.0:1789760752.119609:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6091 00000100:00100000:0.0:1789760752.119620:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762364901750 time 762411664154 state 0x2 ref 0 00000001:00080000:1.0:1789760752.121106:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973380 is committed 00000100:00100000:1.0:1789760752.127558:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760752.127657:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493758976 00000100:00100000:3.0:1789760752.127669:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493758976 00000100:00100000:3.0:1789760752.127720:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762364901750 time 762411664154 state 0x0 ref 1 00000100:00100000:3.0:1789760752.127735:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762411924570 time 762411664154 state 0x1 ref 1 00000100:00100000:3.0:1789760752.127814:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762411924570 time 762468849499 state 0x0 ref 1 00000100:00100000:3.0:1789760752.127837:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493758976:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760752.127888:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24829952, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:3.0:1789760752.127897:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05bec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.127912:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760752.127916:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.127922:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05bec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.127946:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05bec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.186302:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760752.187474:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24829952 -> 24834048 00010000:00010000:3.0:1789760752.187492:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05bec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760752.187499:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973381 is committed 00010000:00010000:3.0:1789760752.187504:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05bec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.187512:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05bec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.187520:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760752.187525:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05bec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.187534:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05bec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.187559:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05bec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.187568:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05bec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.187662:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493758976:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59830us (60108us total) trans 4294973381 rc 0/0 00000100:00100000:3.0:1789760752.187672:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6092 00000100:00100000:3.0:1789760752.187683:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762411924570 time 762468849499 state 0x2 ref 0 00000100:00100000:1.0:1789760752.190502:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760752.190814:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493759104 00000100:00100000:3.0:1789760752.190823:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493759104 00000100:00100000:3.0:1789760752.190871:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762411924570 time 762468849499 state 0x0 ref 1 00000100:00100000:3.0:1789760752.190951:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762469109915 time 762468849499 state 0x1 ref 1 00000100:00100000:3.0:1789760752.191033:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762469109915 time 762532067995 state 0x0 ref 1 00000100:00100000:3.0:1789760752.191045:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493759104:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760752.191093:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24834048, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:3.0:1789760752.191102:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05bf3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.191118:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760752.191124:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.191130:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05bf3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.191144:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05bf3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760752.230018:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24834048 -> 24838144 00010000:00010000:2.0:1789760752.230038:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05bf3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.230052:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05bf3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.230060:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05bf3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.230068:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760752.230072:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05bf3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.230082:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05bf3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.230101:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05bf3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.230110:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05bf3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760752.230198:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493759104:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39150us (39699us total) trans 4294973382 rc 0/0 00000100:00100000:2.0:1789760752.230208:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6093 00000100:00100000:2.0:1789760752.230218:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762469109915 time 762532067995 state 0x2 ref 0 00000001:00080000:1.0:1789760752.230297:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973382 is committed 00000100:00100000:1.0:1789760752.233664:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760752.233854:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493759232 00000100:00100000:3.0:1789760752.233865:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493759232 00000100:00100000:3.0:1789760752.233929:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762469109915 time 762532067995 state 0x0 ref 1 00000100:00100000:3.0:1789760752.233945:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762532328411 time 762532067995 state 0x1 ref 1 00000100:00100000:3.0:1789760752.234002:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762532328411 time 762575036747 state 0x0 ref 1 00000100:00100000:3.0:1789760752.234015:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493759232:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760752.234063:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24838144, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:3.0:1789760752.234072:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05bfa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.234088:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760752.234093:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.234099:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05bfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.234112:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05bfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.293690:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973383 is committed 00002000:00010000:2.0:1789760752.294901:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24838144 -> 24842240 00010000:00010000:2.0:1789760752.294918:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05bfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.294931:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05bfa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.294940:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05bfa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.294948:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760752.294952:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05bfa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.295000:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05bfa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.295021:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05bfa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.295034:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05bfa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760752.295121:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493759232:12345-192.168.203.60@tcp:4:dd.0 Request processed in 61103us (61462us total) trans 4294973383 rc 0/0 00000100:00100000:2.0:1789760752.295132:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6094 00000100:00100000:2.0:1789760752.295141:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762532328411 time 762575036747 state 0x2 ref 0 00000100:00100000:1.0:1789760752.298130:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760752.298221:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493759360 00000100:00100000:0.0:1789760752.298230:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493759360 00000100:00100000:0.0:1789760752.298281:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762532328411 time 762575036747 state 0x0 ref 1 00000100:00100000:0.0:1789760752.298295:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762575297163 time 762575036747 state 0x1 ref 1 00000100:00100000:0.0:1789760752.298327:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762575297163 time 762639362697 state 0x0 ref 1 00000100:00100000:0.0:1789760752.298340:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493759360:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760752.298386:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24842240, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:0.0:1789760752.298395:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05c01 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.298421:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760752.298425:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.298431:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05c01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.298446:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05c01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.357092:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973384 is committed 00002000:00010000:2.0:1789760752.357310:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24842240 -> 24846336 00010000:00010000:2.0:1789760752.357326:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05c01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.357338:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05c01 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.357346:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05c01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.357353:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760752.357358:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05c01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.357366:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05c01 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.357390:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05c01 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.357409:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05c01 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760752.357561:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493759360:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59218us (59435us total) trans 4294973384 rc 0/0 00000100:00100000:2.0:1789760752.357571:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6095 00000100:00100000:2.0:1789760752.357582:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762575297163 time 762639362697 state 0x2 ref 0 00000100:00100000:1.0:1789760752.364425:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760752.364556:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493759488 00000100:00100000:3.0:1789760752.364566:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493759488 00000100:00100000:3.0:1789760752.364616:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762575297163 time 762639362697 state 0x0 ref 1 00000100:00100000:3.0:1789760752.364630:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762639623113 time 762639362697 state 0x1 ref 1 00000100:00100000:3.0:1789760752.364701:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762639623113 time 762705736009 state 0x0 ref 1 00000100:00100000:3.0:1789760752.364712:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493759488:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760752.364759:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24846336, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:3.0:1789760752.364768:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c08 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.364783:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760752.364788:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.364794:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.364807:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.416413:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973385 is committed 00002000:00010000:3.0:1789760752.416535:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24846336 -> 24850432 00010000:00010000:3.0:1789760752.416551:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.416562:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c08 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.416572:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.416579:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760752.416584:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.416593:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c08 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.416610:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c08 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.416619:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c08 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.416756:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493759488:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52041us (52337us total) trans 4294973385 rc 0/0 00000100:00100000:3.0:1789760752.416766:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6096 00000100:00100000:3.0:1789760752.416776:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762639623113 time 762705736009 state 0x2 ref 0 00000100:00100000:1.0:1789760752.423756:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760752.424432:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493759616 00000100:00100000:2.0:1789760752.424441:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493759616 00000100:00100000:2.0:1789760752.424564:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762639623113 time 762705736009 state 0x0 ref 1 00000100:00100000:2.0:1789760752.424579:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762705996425 time 762705736009 state 0x1 ref 1 00000100:00100000:2.0:1789760752.424644:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762705996425 time 762765679239 state 0x0 ref 1 00000100:00100000:2.0:1789760752.424658:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493759616:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760752.424706:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24850432, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:2.0:1789760752.424715:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05c0f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.424731:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760752.424737:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760752.424743:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05c0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.424757:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05c0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760752.456725:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973386 is committed 00002000:00010000:1.0:1789760752.456823:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24850432 -> 24854528 00010000:00010000:1.0:1789760752.456837:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05c0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.456848:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05c0f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.456857:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05c0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.456864:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760752.456868:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05c0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.456955:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05c0f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.457044:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05c0f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.457058:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05c0f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.457182:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493759616:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32522us (33433us total) trans 4294973386 rc 0/0 00000100:00100000:1.0:1789760752.457194:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6097 00000100:00100000:1.0:1789760752.457204:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762705996425 time 762765679239 state 0x2 ref 0 00000100:00100000:1.0:1789760752.461719:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760752.461843:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493759744 00000100:00100000:2.0:1789760752.461852:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493759744 00000100:00100000:2.0:1789760752.463696:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762705996425 time 762765679239 state 0x0 ref 1 00000100:00100000:2.0:1789760752.463720:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762765939655 time 762765679239 state 0x1 ref 1 00000100:00100000:2.0:1789760752.463764:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762765939655 time 762804799409 state 0x0 ref 1 00000100:00100000:2.0:1789760752.463787:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493759744:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760752.463842:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24854528, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:2.0:1789760752.463851:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05c16 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.463867:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760752.463872:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760752.464119:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05c16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.464135:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05c16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760752.491339:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973387 is committed 00002000:00010000:0.0:1789760752.491521:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24854528 -> 24858624 00010000:00010000:0.0:1789760752.491535:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05c16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.491548:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05c16 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.491558:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05c16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.491570:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760752.491574:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05c16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.491584:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05c16 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.491604:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05c16 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.491612:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05c16 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.491739:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493759744:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27958us (30023us total) trans 4294973387 rc 0/0 00000100:00100000:0.0:1789760752.491750:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6098 00000100:00100000:0.0:1789760752.491759:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762765939655 time 762804799409 state 0x2 ref 0 00000100:00100000:0.0:1789760752.495864:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760752.496118:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493759872 00000100:00100000:2.0:1789760752.496127:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493759872 00000100:00100000:2.0:1789760752.496186:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762765939655 time 762804799409 state 0x0 ref 1 00000100:00100000:2.0:1789760752.496202:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762805059825 time 762804799409 state 0x1 ref 1 00000100:00100000:2.0:1789760752.496269:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762805059825 time 762837304414 state 0x0 ref 1 00000100:00100000:2.0:1789760752.496282:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493759872:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760752.496330:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24858624, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:2.0:1789760752.496340:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05c1d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.496356:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760752.496360:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760752.496367:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05c1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.496382:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05c1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760752.530629:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24858624 -> 24862720 00010000:00010000:2.0:1789760752.530647:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05c1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.530665:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05c1d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.530673:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05c1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.530681:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760752.530685:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05c1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.530693:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05c1d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.530711:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05c1d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.530718:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05c1d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760752.530815:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493759872:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34530us (35147us total) trans 4294973388 rc 0/0 00000100:00100000:2.0:1789760752.530825:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6099 00000100:00100000:2.0:1789760752.530835:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762805059825 time 762837304414 state 0x2 ref 0 00000001:00080000:1.0:1789760752.533360:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973388 is committed 00000100:00100000:1.0:1789760752.534752:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760752.534852:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493760000 00000100:00100000:3.0:1789760752.534861:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493760000 00000100:00100000:3.0:1789760752.535039:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762805059825 time 762837304414 state 0x0 ref 1 00000100:00100000:3.0:1789760752.535055:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762837564830 time 762837304414 state 0x1 ref 1 00000100:00100000:3.0:1789760752.535130:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762837564830 time 762876163777 state 0x0 ref 1 00000100:00100000:3.0:1789760752.535144:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493760000:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760752.535194:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24862720, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:3.0:1789760752.535204:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c24 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.535219:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760752.535225:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.535231:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.535247:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760752.565190:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973389 is committed 00002000:00010000:3.0:1789760752.565338:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24862720 -> 24866816 00010000:00010000:3.0:1789760752.565353:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.565365:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c24 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.565375:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.565382:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760752.565388:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.565408:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c24 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.565427:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c24 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.565435:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c24 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.565537:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493760000:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30391us (30786us total) trans 4294973389 rc 0/0 00000100:00100000:3.0:1789760752.565547:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6100 00000100:00100000:3.0:1789760752.565556:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762837564830 time 762876163777 state 0x2 ref 0 00000100:00100000:0.0:1789760752.581133:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760752.581234:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493760128 00000100:00100000:1.0:1789760752.581244:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493760128 00000100:00100000:1.0:1789760752.581302:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762837564830 time 762876163777 state 0x0 ref 1 00000100:00100000:1.0:1789760752.581318:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762876424193 time 762876163777 state 0x1 ref 1 00000100:00100000:1.0:1789760752.581353:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762876424193 time 762922388284 state 0x0 ref 1 00000100:00100000:1.0:1789760752.581366:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493760128:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760752.581424:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24866816, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xc007 00010000:00010000:1.0:1789760752.581434:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05c2b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.581455:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760752.581459:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.581466:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05c2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.581479:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05c2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760752.653808:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973390 is committed 00002000:00010000:1.0:1789760752.653918:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24866816 -> 24870912 00002000:00010000:1.0:1789760752.653923:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760751 -> 1789760752 00002000:00010000:1.0:1789760752.653926:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760751 -> 1789760752 00010000:00010000:1.0:1789760752.653939:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05c2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.653952:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05c2b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.653982:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05c2b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.653990:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760752.653994:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05c2b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.654003:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05c2b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.654023:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05c2b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.654031:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05c2b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.654131:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493760128:12345-192.168.203.60@tcp:4:dd.0 Request processed in 72763us (73003us total) trans 4294973390 rc 0/0 00000100:00100000:1.0:1789760752.654142:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6101 00000100:00100000:1.0:1789760752.654152:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762876424193 time 762922388284 state 0x2 ref 0 00000100:00100000:1.0:1789760752.659022:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760752.659072:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493760256 00000100:00100000:0.0:1789760752.659083:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493760256 00000100:00100000:0.0:1789760752.659141:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762876424193 time 762922388284 state 0x0 ref 1 00000100:00100000:0.0:1789760752.659161:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762922648700 time 762922388284 state 0x1 ref 1 00000100:00100000:0.0:1789760752.659267:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762922648700 time 763000301552 state 0x0 ref 1 00000100:00100000:0.0:1789760752.659281:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493760256:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760752.659330:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24870912, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:0.0:1789760752.659340:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05c32 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.659357:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760752.659362:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.659368:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05c32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.659383:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05c32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760752.686820:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24870912 -> 24875008 00010000:00010000:1.0:1789760752.686839:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05c32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.686854:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05c32 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.686865:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05c32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.686873:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760752.686879:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05c32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.686896:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05c32 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.686920:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05c32 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.686929:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05c32 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.687055:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493760256:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27771us (28039us total) trans 4294973391 rc 0/0 00000100:00100000:1.0:1789760752.687064:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6102 00000100:00100000:1.0:1789760752.687074:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762922648700 time 763000301552 state 0x2 ref 0 00000001:00080000:3.0:1789760752.688573:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973391 is committed 00000100:00100000:0.0:1789760752.696611:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760752.696718:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493760384 00000100:00100000:2.0:1789760752.696729:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493760384 00000100:00100000:2.0:1789760752.696789:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 762922648700 time 763000301552 state 0x0 ref 1 00000100:00100000:2.0:1789760752.696805:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763000561968 time 763000301552 state 0x1 ref 1 00000100:00100000:2.0:1789760752.696891:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763000561968 time 763037926258 state 0x0 ref 1 00000100:00100000:2.0:1789760752.696904:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493760384:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760752.696948:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24875008, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:2.0:1789760752.696958:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05c39 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.697007:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:2.0:1789760752.697011:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760752.697018:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05c39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.697031:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05c39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760752.738313:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973392 is committed 00002000:00010000:0.0:1789760752.740574:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24875008 -> 24879104 00010000:00010000:0.0:1789760752.740597:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05c39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.740611:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05c39 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.740619:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05c39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.740633:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760752.740638:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05c39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.740647:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05c39 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.740669:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05c39 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.740676:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05c39 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.740850:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493760384:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43941us (44241us total) trans 4294973392 rc 0/0 00000100:00100000:0.0:1789760752.740862:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6103 00000100:00100000:0.0:1789760752.740871:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763000561968 time 763037926258 state 0x2 ref 0 00000100:00100000:1.0:1789760752.744117:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760752.744196:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760752.744294:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493760512 00000100:00100000:2.0:1789760752.744304:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493760512 00000100:00100000:2.0:1789760752.744363:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 403 00000100:00100000:2.0:1789760752.744373:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d48850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2370:x1876699493760512:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:0.0:1789760752.744461:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493760640 00000100:00100000:0.0:1789760752.744465:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493760640 00000100:00100000:0.0:1789760752.744509:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 492 00000100:00100000:0.0:1789760752.744518:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2370:x1876699493760640:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:2.0:1789760752.744811:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d48850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2370:x1876699493760512:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 437us (613us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760752.744820:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 403 00000100:00100000:1.0:1789760752.745176:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760752.745362:0:10519:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493760768 00000100:00100000:2.0:1789760752.745368:0:10519:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493760768 00000100:00100000:2.0:1789760752.745415:0:10519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 493 00000100:00100000:2.0:1789760752.745423:0:10519:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2370:x1876699493760768:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:2.0:1789760752.745522:0:10519:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2370:x1876699493760768:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 100us (345us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760752.745529:0:10519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 493 00000100:00100000:0.0:1789760752.746031:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2370:x1876699493760640:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 1512us (1920us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760752.746039:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 492 00000100:00080000:1.0:1789760752.746609:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760752.753441:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760752.754340:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493760896 00000100:00100000:0.0:1789760752.754351:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493760896 00000100:00100000:0.0:1789760752.754414:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763000561968 time 763037926258 state 0x0 ref 1 00000100:00100000:0.0:1789760752.754429:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763038186674 time 763037926258 state 0x1 ref 1 00000100:00100000:0.0:1789760752.754459:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763038186674 time 763095494482 state 0x0 ref 1 00000100:00100000:0.0:1789760752.754470:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493760896:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760752.754515:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24879104, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:0.0:1789760752.754523:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05c40 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.754539:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760752.754543:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.754549:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05c40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.754563:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05c40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760752.835222:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24879104 -> 24883200 00010000:00010000:0.0:1789760752.835244:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05c40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.835256:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05c40 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.835270:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05c40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.835277:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760752.835282:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05c40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.835291:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05c40 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.835310:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05c40 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.835318:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05c40 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760752.835599:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973393 is committed 00000100:00100000:0.0:1789760752.836012:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493760896:12345-192.168.203.60@tcp:4:dd.0 Request processed in 81537us (82575us total) trans 4294973393 rc 0/0 00000100:00100000:0.0:1789760752.836023:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6104 00000100:00100000:0.0:1789760752.836032:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763038186674 time 763095494482 state 0x2 ref 0 00000100:00100000:2.0:1789760752.850440:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760752.852727:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493761024 00000100:00100000:1.0:1789760752.852741:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493761024 00000100:00100000:1.0:1789760752.852805:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763038186674 time 763095494482 state 0x0 ref 1 00000100:00100000:1.0:1789760752.852821:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763095754898 time 763095494482 state 0x1 ref 1 00000100:00100000:1.0:1789760752.852846:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763095754898 time 763193881891 state 0x0 ref 1 00000100:00100000:1.0:1789760752.852859:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493761024:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760752.852905:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24883200, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:1.0:1789760752.852914:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05c47 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.852931:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760752.852935:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.852941:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05c47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.852956:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05c47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760752.896495:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973394 is committed 00002000:00010000:0.0:1789760752.896600:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24883200 -> 24887296 00010000:00010000:0.0:1789760752.896615:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05c47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.896635:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05c47 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.896644:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05c47 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.896651:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760752.896656:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05c47 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.896664:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05c47 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.896682:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05c47 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.896690:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05c47 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.896782:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493761024:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43921us (46349us total) trans 4294973394 rc 0/0 00000100:00100000:0.0:1789760752.896791:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6105 00000100:00100000:0.0:1789760752.896800:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763095754898 time 763193881891 state 0x2 ref 0 00000100:00100000:3.0:1789760752.899765:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760752.899882:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493761152 00000100:00100000:1.0:1789760752.899892:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493761152 00000100:00100000:1.0:1789760752.899951:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763095754898 time 763193881891 state 0x0 ref 1 00000100:00100000:1.0:1789760752.900002:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763194142307 time 763193881891 state 0x1 ref 1 00000100:00100000:1.0:1789760752.900025:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763194142307 time 763241061434 state 0x0 ref 1 00000100:00100000:1.0:1789760752.900038:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493761152:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760752.900089:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24887296, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:1.0:1789760752.900098:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05c4e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.900113:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760752.900118:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.900123:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05c4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.900136:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05c4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760752.934516:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973395 is committed 00002000:00010000:0.0:1789760752.934707:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24887296 -> 24891392 00010000:00010000:0.0:1789760752.934724:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05c4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.934738:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05c4e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.934748:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05c4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.934757:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760752.934762:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05c4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.934772:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05c4e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.934790:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05c4e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.934798:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05c4e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.935244:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493761152:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35203us (35486us total) trans 4294973395 rc 0/0 00000100:00100000:0.0:1789760752.935254:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6106 00000100:00100000:0.0:1789760752.935265:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763194142307 time 763241061434 state 0x2 ref 0 00000100:00100000:2.0:1789760752.941653:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760752.941766:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493761280 00000100:00100000:0.0:1789760752.941777:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493761280 00000100:00100000:0.0:1789760752.941827:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763194142307 time 763241061434 state 0x0 ref 1 00000100:00100000:0.0:1789760752.941843:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763241321850 time 763241061434 state 0x1 ref 1 00000100:00100000:0.0:1789760752.941879:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763241321850 time 763282914225 state 0x0 ref 1 00000100:00100000:0.0:1789760752.941891:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493761280:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760752.941940:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24891392, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:0.0:1789760752.941949:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05c55 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.942266:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760752.942271:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.942279:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05c55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.942294:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05c55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760752.972933:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973396 is committed 00002000:00010000:2.0:1789760752.973020:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24891392 -> 24895488 00010000:00010000:2.0:1789760752.973034:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05c55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.973047:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05c55 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.973056:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05c55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.973063:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760752.973069:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05c55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.973078:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05c55 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.973098:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05c55 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.973106:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e05c55 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760752.973181:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493761280:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31288us (31534us total) trans 4294973396 rc 0/0 00000100:00100000:2.0:1789760752.973190:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6107 00000100:00100000:2.0:1789760752.973200:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763241321850 time 763282914225 state 0x2 ref 0 00000100:00100000:3.0:1789760752.982528:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760752.982617:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493761408 00000100:00100000:0.0:1789760752.982627:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493761408 00000100:00100000:0.0:1789760752.982680:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763241321850 time 763282914225 state 0x0 ref 1 00000100:00100000:0.0:1789760752.982696:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763283174641 time 763282914225 state 0x1 ref 1 00000100:00100000:0.0:1789760752.982763:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763283174641 time 763323797652 state 0x0 ref 1 00000100:00100000:0.0:1789760752.982775:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493761408:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760752.982821:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24895488, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:0.0:1789760752.982830:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05c5c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.982846:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760752.982850:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.982856:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05c5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.982870:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05c5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760753.005531:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24895488 -> 24899584 00010000:00010000:1.0:1789760753.005549:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05c5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.005562:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05c5c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.005571:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05c5c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.005579:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760753.005584:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05c5c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.005593:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05c5c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.005616:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05c5c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.005624:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e05c5c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.005711:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493761408:12345-192.168.203.60@tcp:4:dd.0 Request processed in 22933us (23187us total) trans 4294973397 rc 0/0 00000100:00100000:1.0:1789760753.005720:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6108 00000100:00100000:1.0:1789760753.005730:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763283174641 time 763323797652 state 0x2 ref 0 00000001:00080000:2.0:1789760753.008585:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973397 is committed 00000100:00100000:2.0:1789760753.009264:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760753.009353:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493761536 00000100:00100000:1.0:1789760753.009362:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493761536 00000100:00100000:1.0:1789760753.009473:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763283174641 time 763323797652 state 0x0 ref 1 00000100:00100000:1.0:1789760753.009488:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763324058068 time 763323797652 state 0x1 ref 1 00000100:00100000:1.0:1789760753.009520:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763324058068 time 763350555836 state 0x0 ref 1 00000100:00100000:1.0:1789760753.009534:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493761536:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760753.009581:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24899584, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:1.0:1789760753.009589:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05c63 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.009605:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760753.009610:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.009616:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05c63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.009631:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05c63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760753.010500:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760753.041886:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973398 is committed 00002000:00010000:3.0:1789760753.042073:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24899584 -> 24903680 00010000:00010000:3.0:1789760753.042088:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05c63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.042101:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05c63 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.042110:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05c63 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.042117:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760753.042121:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05c63 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.042129:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05c63 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.042149:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05c63 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.042158:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05c63 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.042263:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493761536:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32727us (32999us total) trans 4294973398 rc 0/0 00000100:00100000:3.0:1789760753.042273:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6109 00000100:00100000:3.0:1789760753.042283:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763324058068 time 763350555836 state 0x2 ref 0 00000100:00100000:3.0:1789760753.045904:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760753.046023:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493761664 00000100:00100000:1.0:1789760753.046033:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493761664 00000100:00100000:1.0:1789760753.046093:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763324058068 time 763350555836 state 0x0 ref 1 00000100:00100000:1.0:1789760753.046110:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763350816252 time 763350555836 state 0x1 ref 1 00000100:00100000:1.0:1789760753.046148:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763350816252 time 763387182913 state 0x0 ref 1 00000100:00100000:1.0:1789760753.046161:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493761664:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760753.046214:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24903680, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:1.0:1789760753.046223:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c6a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.046238:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760753.046243:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.046249:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.046264:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760753.092584:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973399 is committed 00002000:00010000:1.0:1789760753.092599:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24903680 -> 24907776 00010000:00010000:1.0:1789760753.092615:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.092628:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c6a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.092638:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.092645:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760753.092651:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.092660:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c6a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.092679:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c6a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.092687:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c6a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.092778:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493761664:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46614us (46877us total) trans 4294973399 rc 0/0 00000100:00100000:1.0:1789760753.092793:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6110 00000100:00100000:1.0:1789760753.092802:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763350816252 time 763387182913 state 0x2 ref 0 00000100:00100000:0.0:1789760753.095912:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760753.095995:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493761792 00000100:00100000:3.0:1789760753.096008:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493761792 00000100:00100000:3.0:1789760753.096051:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763350816252 time 763387182913 state 0x0 ref 1 00000100:00100000:3.0:1789760753.096062:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763387443329 time 763387182913 state 0x1 ref 1 00000100:00100000:3.0:1789760753.096083:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763387443329 time 763437119559 state 0x0 ref 1 00000100:00100000:3.0:1789760753.096091:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493761792:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760753.096485:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24907776, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:3.0:1789760753.096495:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c71 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.096510:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760753.096515:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.096585:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.096601:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760753.135322:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973400 is committed 00002000:00010000:2.0:1789760753.135430:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24907776 -> 24911872 00010000:00010000:2.0:1789760753.135511:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.135524:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c71 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.135532:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c71 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.135543:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760753.135548:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c71 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.135557:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c71 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.135576:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c71 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.135583:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05c71 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.135711:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493761792:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39614us (39802us total) trans 4294973400 rc 0/0 00000100:00100000:2.0:1789760753.135722:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6111 00000100:00100000:2.0:1789760753.135732:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763387443329 time 763437119559 state 0x2 ref 0 00000100:00100000:3.0:1789760753.139167:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760753.139384:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493761920 00000100:00100000:0.0:1789760753.139402:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493761920 00000100:00100000:0.0:1789760753.139472:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763387443329 time 763437119559 state 0x0 ref 1 00000100:00100000:0.0:1789760753.139486:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763437379975 time 763437119559 state 0x1 ref 1 00000100:00100000:0.0:1789760753.139565:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763437379975 time 763480599583 state 0x0 ref 1 00000100:00100000:0.0:1789760753.139578:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493761920:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760753.139629:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24911872, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:0.0:1789760753.139638:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05c78 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.139653:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760753.139657:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760753.139664:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05c78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.139678:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05c78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760753.164691:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24911872 -> 24915968 00010000:00010000:0.0:1789760753.164711:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05c78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.164724:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05c78 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.164735:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05c78 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.164743:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760753.164747:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05c78 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.164757:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05c78 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.164777:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05c78 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.164785:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05c78 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760753.164870:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493761920:12345-192.168.203.60@tcp:4:dd.0 Request processed in 25289us (25708us total) trans 4294973401 rc 0/0 00000100:00100000:0.0:1789760753.164880:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6112 00000100:00100000:0.0:1789760753.164889:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763437379975 time 763480599583 state 0x2 ref 0 00000001:00080000:0.0:1789760753.164925:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973401 is committed 00000100:00100000:2.0:1789760753.171823:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760753.171921:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493762048 00000100:00100000:1.0:1789760753.171930:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493762048 00000100:00100000:1.0:1789760753.172029:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763437379975 time 763480599583 state 0x0 ref 1 00000100:00100000:1.0:1789760753.172046:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763480859999 time 763480599583 state 0x1 ref 1 00000100:00100000:1.0:1789760753.172075:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763480859999 time 763513110913 state 0x0 ref 1 00000100:00100000:1.0:1789760753.172089:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493762048:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760753.172136:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24915968, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:1.0:1789760753.172146:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c7f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.172162:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760753.172167:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.172173:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.172200:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760753.206516:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973402 is committed 00002000:00010000:1.0:1789760753.206643:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24915968 -> 24920064 00010000:00010000:1.0:1789760753.206658:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.206671:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c7f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.206680:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c7f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.206687:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760753.206691:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c7f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.206700:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c7f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.206719:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c7f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.206728:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c7f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.206828:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493762048:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34737us (35009us total) trans 4294973402 rc 0/0 00000100:00100000:1.0:1789760753.206839:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6113 00000100:00100000:1.0:1789760753.206848:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763480859999 time 763513110913 state 0x2 ref 0 00010000:00010000:0.0:1789760753.210177:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760753.215498:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760753.215568:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493762176 00000100:00100000:1.0:1789760753.215578:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493762176 00000100:00100000:1.0:1789760753.215634:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763480859999 time 763513110913 state 0x0 ref 1 00000100:00100000:1.0:1789760753.215647:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763513371329 time 763513110913 state 0x1 ref 1 00000100:00100000:1.0:1789760753.215671:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763513371329 time 763556707559 state 0x0 ref 1 00000100:00100000:1.0:1789760753.215682:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493762176:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760753.215724:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24920064, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:1.0:1789760753.215732:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c86 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.215747:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760753.215752:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.215758:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.215771:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760753.267491:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973403 is committed 00002000:00010000:1.0:1789760753.267675:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24920064 -> 24924160 00010000:00010000:1.0:1789760753.267692:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.267705:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c86 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.267721:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c86 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.267728:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760753.267732:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c86 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.267740:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c86 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.267757:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c86 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.267764:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05c86 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.267854:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493762176:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52168us (52359us total) trans 4294973403 rc 0/0 00000100:00100000:1.0:1789760753.267864:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6114 00000100:00100000:1.0:1789760753.267873:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763513371329 time 763556707559 state 0x2 ref 0 00000100:00100000:0.0:1789760753.272373:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760753.272562:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493762304 00000100:00100000:2.0:1789760753.272573:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493762304 00000100:00100000:2.0:1789760753.272628:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763513371329 time 763556707559 state 0x0 ref 1 00000100:00100000:2.0:1789760753.272643:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763556967975 time 763556707559 state 0x1 ref 1 00000100:00100000:2.0:1789760753.272708:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763556967975 time 763613743115 state 0x0 ref 1 00000100:00100000:2.0:1789760753.272730:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493762304:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760753.272781:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24924160, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:2.0:1789760753.272790:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05c8d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.272806:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:2.0:1789760753.272811:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.272817:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05c8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.272832:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05c8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760753.322657:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973404 is committed 00002000:00010000:0.0:1789760753.323043:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24924160 -> 24928256 00010000:00010000:0.0:1789760753.323062:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05c8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.323075:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05c8d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.323083:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05c8d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.323096:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760753.323101:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05c8d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.323108:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05c8d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.323128:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05c8d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.323135:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05c8d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760753.323234:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493762304:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50501us (50867us total) trans 4294973404 rc 0/0 00000100:00100000:0.0:1789760753.323245:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6115 00000100:00100000:0.0:1789760753.323254:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763556967975 time 763613743115 state 0x2 ref 0 00000100:00100000:2.0:1789760753.329464:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760753.329588:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493762432 00000100:00100000:0.0:1789760753.329598:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493762432 00000100:00100000:0.0:1789760753.329652:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763556967975 time 763613743115 state 0x0 ref 1 00000100:00100000:0.0:1789760753.329667:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763614003531 time 763613743115 state 0x1 ref 1 00000100:00100000:0.0:1789760753.329701:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763614003531 time 763670736259 state 0x0 ref 1 00000100:00100000:0.0:1789760753.329799:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493762432:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760753.329932:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24928256, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:0.0:1789760753.329941:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05c94 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.329958:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760753.330088:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760753.330095:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05c94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.330110:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05c94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.375564:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973405 is committed 00002000:00010000:1.0:1789760753.375933:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24928256 -> 24932352 00010000:00010000:1.0:1789760753.375948:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05c94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.375996:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05c94 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.376019:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05c94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.376025:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760753.376036:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05c94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.376044:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05c94 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.376063:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05c94 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.376071:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05c94 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.376172:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493762432:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46373us (46713us total) trans 4294973405 rc 0/0 00000100:00100000:1.0:1789760753.376183:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6116 00000100:00100000:1.0:1789760753.376193:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763614003531 time 763670736259 state 0x2 ref 0 00000100:00100000:1.0:1789760753.379832:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760753.379883:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493762560 00000100:00100000:3.0:1789760753.379893:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493762560 00000100:00100000:3.0:1789760753.379946:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763614003531 time 763670736259 state 0x0 ref 1 00000100:00100000:3.0:1789760753.379958:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763670996675 time 763670736259 state 0x1 ref 1 00000100:00100000:3.0:1789760753.380012:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763670996675 time 763721047017 state 0x0 ref 1 00000100:00100000:3.0:1789760753.380031:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493762560:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760753.380082:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24932352, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:3.0:1789760753.380090:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05c9b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.380105:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760753.380146:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.380154:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05c9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.380170:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05c9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.430000:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973406 is committed 00002000:00010000:3.0:1789760753.430065:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24932352 -> 24936448 00010000:00010000:3.0:1789760753.430080:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05c9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.430092:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05c9b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.430107:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05c9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.430115:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760753.430120:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05c9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.430129:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05c9b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.430149:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05c9b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.430158:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05c9b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.430251:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493762560:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50218us (50421us total) trans 4294973406 rc 0/0 00000100:00100000:3.0:1789760753.430261:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6117 00000100:00100000:3.0:1789760753.430270:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763670996675 time 763721047017 state 0x2 ref 0 00000100:00100000:0.0:1789760753.434201:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760753.434235:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493762688 00000100:00100000:1.0:1789760753.434245:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493762688 00000100:00100000:1.0:1789760753.434305:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763670996675 time 763721047017 state 0x0 ref 1 00000100:00100000:1.0:1789760753.434320:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763721307433 time 763721047017 state 0x1 ref 1 00000100:00100000:1.0:1789760753.434353:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763721307433 time 763775387745 state 0x0 ref 1 00000100:00100000:1.0:1789760753.434365:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493762688:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760753.434423:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24936448, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:1.0:1789760753.434432:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ca2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.434446:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760753.434451:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.434457:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ca2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.434470:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ca2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.479897:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973407 is committed 00002000:00010000:2.0:1789760753.482056:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24936448 -> 24940544 00010000:00010000:2.0:1789760753.482077:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ca2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.482090:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ca2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.482100:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ca2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.482507:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760753.482515:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ca2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.482525:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ca2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.482546:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ca2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.482555:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ca2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.483066:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493762688:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48698us (48869us total) trans 4294973407 rc 0/0 00000100:00100000:2.0:1789760753.483077:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6118 00000100:00100000:2.0:1789760753.483086:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763721307433 time 763775387745 state 0x2 ref 0 00000100:00100000:0.0:1789760753.487778:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760753.487867:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493762816 00000100:00100000:2.0:1789760753.487946:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493762816 00000100:00100000:3.0:1789760753.488188:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763721307433 time 763775387745 state 0x0 ref 1 00000100:00100000:3.0:1789760753.488204:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763775648161 time 763775387745 state 0x1 ref 1 00000100:00100000:3.0:1789760753.488273:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763775648161 time 763829308025 state 0x0 ref 1 00000100:00100000:3.0:1789760753.488288:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493762816:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760753.488335:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24940544, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:3.0:1789760753.488343:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05ca9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.488359:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760753.488363:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.488369:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05ca9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.488384:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05ca9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760753.528507:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24940544 -> 24944640 00010000:00010000:1.0:1789760753.528526:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05ca9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.528539:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05ca9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.528555:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05ca9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.528564:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760753.528569:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05ca9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.528579:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05ca9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.528599:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05ca9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.528607:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05ca9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.528706:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493762816:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40418us (40934us total) trans 4294973408 rc 0/0 00000100:00100000:1.0:1789760753.528717:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6119 00000100:00100000:1.0:1789760753.528727:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763775648161 time 763829308025 state 0x2 ref 0 00000001:00080000:3.0:1789760753.528996:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973408 is committed 00000100:00100000:2.0:1789760753.531112:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760753.531202:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc33ea300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3304:1876699519254912:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760753.531277:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760753.534856:0:6691:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519254912 00000100:00100000:1.0:1789760753.534870:0:6691:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519254912 00000100:00100000:1.0:1789760753.534952:0:6691:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 204 00000100:00100000:1.0:1789760753.534976:0:6691:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519254912:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760753.535005:0:6691:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526843904 pending 0 free 3833593856 avail 3813670912 00000100:00080000:2.0:1789760753.535045:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760753.535080:0:6691:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519254912:12345-0@lo:13:osp-pre-0-0.0 Request processed in 120us (3802us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760753.535086:0:6691:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 204 00000100:00100000:2.0:1789760753.535948:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc33ea300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3304:1876699519254912:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760753.543146:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760753.543753:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493762944 00000100:00100000:0.0:1789760753.543763:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493762944 00000100:00100000:0.0:1789760753.543818:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763775648161 time 763829308025 state 0x0 ref 1 00000100:00100000:0.0:1789760753.543833:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763829568441 time 763829308025 state 0x1 ref 1 00000100:00100000:0.0:1789760753.543866:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763829568441 time 763884901498 state 0x0 ref 1 00000100:00100000:0.0:1789760753.543878:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493762944:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760753.543926:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24944640, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:0.0:1789760753.543936:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05cb0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.543951:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760753.544005:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760753.544013:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05cb0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.544027:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05cb0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760753.610324:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973409 is committed 00002000:00010000:0.0:1789760753.610569:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24944640 -> 24948736 00010000:00010000:0.0:1789760753.610583:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05cb0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.610595:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05cb0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.610603:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05cb0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.610610:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760753.610622:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05cb0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.610631:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05cb0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.610655:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05cb0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.610663:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05cb0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760753.610753:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493762944:12345-192.168.203.60@tcp:4:dd.0 Request processed in 66872us (67613us total) trans 4294973409 rc 0/0 00000100:00100000:0.0:1789760753.610763:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6120 00000100:00100000:0.0:1789760753.610773:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763829568441 time 763884901498 state 0x2 ref 0 00000100:00100000:0.0:1789760753.631917:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760753.632032:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493763072 00000100:00100000:1.0:1789760753.632042:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493763072 00000100:00100000:1.0:1789760753.632092:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763829568441 time 763884901498 state 0x0 ref 1 00000100:00100000:1.0:1789760753.632107:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763885161914 time 763884901498 state 0x1 ref 1 00000100:00100000:1.0:1789760753.632137:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763885161914 time 763973172912 state 0x0 ref 1 00000100:00100000:1.0:1789760753.632150:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7aa7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493763072:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760753.632198:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24948736, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xc007 00010000:00010000:1.0:1789760753.632206:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05cb7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.632221:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760753.632226:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.632232:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05cb7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.632245:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05cb7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.666146:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973410 is committed 00002000:00010000:1.0:1789760753.666272:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24948736 -> 24952832 00002000:00010000:1.0:1789760753.666277:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760752 -> 1789760753 00002000:00010000:1.0:1789760753.666279:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760752 -> 1789760753 00010000:00010000:1.0:1789760753.666291:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05cb7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.666304:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05cb7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.666313:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05cb7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.666321:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760753.666326:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05cb7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.666334:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05cb7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.666356:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05cb7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.666363:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05cb7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.666477:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7aa7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493763072:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34324us (34565us total) trans 4294973410 rc 0/0 00000100:00100000:1.0:1789760753.666496:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6121 00000100:00100000:1.0:1789760753.666506:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763885161914 time 763973172912 state 0x2 ref 0 00000100:00100000:3.0:1789760753.670527:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760753.670682:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493763200 00000100:00100000:2.0:1789760753.670691:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493763200 00000100:00100000:2.0:1789760753.670748:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763885161914 time 763973172912 state 0x0 ref 1 00000100:00100000:2.0:1789760753.670765:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763973433328 time 763973172912 state 0x1 ref 1 00000100:00100000:2.0:1789760753.670834:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763973433328 time 764011869668 state 0x0 ref 1 00000100:00100000:2.0:1789760753.670847:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493763200:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760753.670894:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24952832, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:2.0:1789760753.670903:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05cbe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.670918:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:2.0:1789760753.670922:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.670928:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05cbe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.670941:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05cbe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760753.694500:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973411 is committed 00002000:00010000:3.0:1789760753.694628:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24952832 -> 24956928 00010000:00010000:3.0:1789760753.694643:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05cbe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.694656:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05cbe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.694665:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05cbe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.694674:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760753.694678:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05cbe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.694688:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05cbe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.694706:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05cbe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.694714:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05cbe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.694809:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493763200:12345-192.168.203.60@tcp:4:dd.0 Request processed in 23959us (24287us total) trans 4294973411 rc 0/0 00000100:00100000:3.0:1789760753.694819:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6122 00000100:00100000:3.0:1789760753.694829:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763973433328 time 764011869668 state 0x2 ref 0 00000100:00100000:3.0:1789760753.701940:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760753.702051:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493763328 00000100:00100000:1.0:1789760753.702062:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493763328 00000100:00100000:1.0:1789760753.702120:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 763973433328 time 764011869668 state 0x0 ref 1 00000100:00100000:1.0:1789760753.702135:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764012130084 time 764011869668 state 0x1 ref 1 00000100:00100000:1.0:1789760753.702167:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764012130084 time 764043202677 state 0x0 ref 1 00000100:00100000:1.0:1789760753.702179:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493763328:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760753.702225:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24956928, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:1.0:1789760753.702234:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05cc5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.702248:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:1.0:1789760753.702252:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.702269:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05cc5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.702283:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05cc5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760753.756251:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24956928 -> 24961024 00010000:00010000:1.0:1789760753.756270:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05cc5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.756284:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05cc5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.756293:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05cc5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.756301:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760753.756305:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05cc5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.756315:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05cc5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.756334:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05cc5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.756341:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05cc5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760753.756595:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973412 is committed 00000100:00100000:1.0:1789760753.756648:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493763328:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54264us (54510us total) trans 4294973412 rc 0/0 00000100:00100000:1.0:1789760753.756661:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6123 00000100:00100000:1.0:1789760753.756670:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764012130084 time 764043202677 state 0x2 ref 0 00000100:00100000:1.0:1789760753.762203:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760753.762289:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493763456 00000100:00100000:0.0:1789760753.762299:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493763456 00000100:00100000:0.0:1789760753.762352:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764012130084 time 764043202677 state 0x0 ref 1 00000100:00100000:0.0:1789760753.762367:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764043463093 time 764043202677 state 0x1 ref 1 00000100:00100000:0.0:1789760753.762404:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764043463093 time 764103439951 state 0x0 ref 1 00000100:00100000:0.0:1789760753.762417:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493763456:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760753.762473:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24961024, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:0.0:1789760753.762483:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05ccc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.762502:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760753.762506:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760753.762513:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05ccc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.762527:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05ccc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760753.799844:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973413 is committed 00002000:00010000:3.0:1789760753.800011:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24961024 -> 24965120 00010000:00010000:3.0:1789760753.800025:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05ccc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.800038:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05ccc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.800052:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05ccc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.800060:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760753.800064:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05ccc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.800074:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05ccc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.800095:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05ccc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.800103:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05ccc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.800193:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493763456:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37772us (37993us total) trans 4294973413 rc 0/0 00000100:00100000:3.0:1789760753.800204:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6124 00000100:00100000:3.0:1789760753.800214:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764043463093 time 764103439951 state 0x2 ref 0 00000100:00100000:3.0:1789760753.803179:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760753.803265:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493763584 00000100:00100000:2.0:1789760753.803273:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493763584 00000100:00100000:2.0:1789760753.803324:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764043463093 time 764103439951 state 0x0 ref 1 00000100:00100000:2.0:1789760753.803338:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764103700367 time 764103439951 state 0x1 ref 1 00000100:00100000:2.0:1789760753.803413:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764103700367 time 764144448036 state 0x0 ref 1 00000100:00100000:2.0:1789760753.803426:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb06450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493763584:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760753.803484:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24965120, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:2.0:1789760753.803492:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cd3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.803507:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:2.0:1789760753.803511:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.803516:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cd3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.803530:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cd3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760753.837642:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24965120 -> 24969216 00010000:00010000:2.0:1789760753.837659:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cd3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.837672:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cd3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.837681:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cd3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.837688:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760753.837692:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cd3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.837703:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cd3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.837720:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cd3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.837727:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cd3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.837810:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb06450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493763584:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34381us (34634us total) trans 4294973414 rc 0/0 00000100:00100000:2.0:1789760753.837820:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6125 00000100:00100000:2.0:1789760753.837829:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764103700367 time 764144448036 state 0x2 ref 0 00000001:00080000:2.0:1789760753.838011:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973414 is committed 00000100:00100000:1.0:1789760753.844326:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760753.844634:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493763712 00000100:00100000:3.0:1789760753.844645:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493763712 00000100:00100000:3.0:1789760753.844697:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764103700367 time 764144448036 state 0x0 ref 1 00000100:00100000:3.0:1789760753.844713:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764144708452 time 764144448036 state 0x1 ref 1 00000100:00100000:3.0:1789760753.844800:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764144708452 time 764185834596 state 0x0 ref 1 00000100:00100000:3.0:1789760753.844814:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493763712:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760753.844860:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24969216, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:3.0:1789760753.845048:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05cda lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.845062:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760753.845066:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.845073:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05cda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.845086:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05cda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760753.871906:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973415 is committed 00002000:00010000:2.0:1789760753.872546:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24969216 -> 24973312 00010000:00010000:2.0:1789760753.872560:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05cda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.872574:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05cda lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.872582:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05cda lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.872593:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760753.872597:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05cda lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.872605:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05cda lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.872622:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05cda lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.872630:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05cda lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.872753:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493763712:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27937us (28433us total) trans 4294973415 rc 0/0 00000100:00100000:2.0:1789760753.872762:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6126 00000100:00100000:2.0:1789760753.872771:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764144708452 time 764185834596 state 0x2 ref 0 00000100:00100000:3.0:1789760753.876502:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760753.876569:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493763840 00000100:00100000:2.0:1789760753.876579:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493763840 00000100:00100000:2.0:1789760753.876634:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764144708452 time 764185834596 state 0x0 ref 1 00000100:00100000:2.0:1789760753.876649:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764186095012 time 764185834596 state 0x1 ref 1 00000100:00100000:2.0:1789760753.876682:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764186095012 time 764217717824 state 0x0 ref 1 00000100:00100000:2.0:1789760753.876697:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493763840:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760753.876742:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24973312, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:2.0:1789760753.876751:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05ce1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.876767:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:2.0:1789760753.876772:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.876778:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05ce1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.876790:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05ce1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760753.909890:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24973312 -> 24977408 00010000:00010000:2.0:1789760753.909911:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05ce1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.909923:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05ce1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.909932:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05ce1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.909940:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760753.909944:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05ce1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.909954:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05ce1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.910003:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05ce1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.910011:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05ce1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.910092:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493763840:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33392us (33595us total) trans 4294973416 rc 0/0 00000100:00100000:2.0:1789760753.910102:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6127 00000100:00100000:2.0:1789760753.910112:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764186095012 time 764217717824 state 0x2 ref 0 00000001:00080000:1.0:1789760753.910642:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973416 is committed 00000100:00100000:1.0:1789760753.913021:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760753.913115:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493763968 00000100:00100000:0.0:1789760753.913125:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493763968 00000100:00100000:0.0:1789760753.913179:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764186095012 time 764217717824 state 0x0 ref 1 00000100:00100000:0.0:1789760753.913195:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764217978240 time 764217717824 state 0x1 ref 1 00000100:00100000:0.0:1789760753.913222:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764217978240 time 764254257172 state 0x0 ref 1 00000100:00100000:0.0:1789760753.913233:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493763968:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760753.913284:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24977408, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:0.0:1789760753.913294:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05ce8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.913310:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760753.913314:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760753.913321:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05ce8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.913341:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05ce8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760753.951652:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24977408 -> 24981504 00010000:00010000:0.0:1789760753.951672:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05ce8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.951685:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05ce8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.951695:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05ce8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.951702:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760753.951707:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05ce8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.951716:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05ce8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.951736:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05ce8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.951744:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05ce8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760753.951797:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973417 is committed 00000100:00100000:0.0:1789760753.951836:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493763968:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38599us (38820us total) trans 4294973417 rc 0/0 00000100:00100000:0.0:1789760753.951847:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6128 00000100:00100000:0.0:1789760753.951856:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764217978240 time 764254257172 state 0x2 ref 0 00000100:00100000:3.0:1789760753.955800:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760753.956213:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493764096 00000100:00100000:0.0:1789760753.956223:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493764096 00000100:00100000:2.0:1789760753.956455:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764217978240 time 764254257172 state 0x0 ref 1 00000100:00100000:2.0:1789760753.956471:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764254517588 time 764254257172 state 0x1 ref 1 00000100:00100000:2.0:1789760753.956533:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764254517588 time 764297568788 state 0x0 ref 1 00000100:00100000:2.0:1789760753.956546:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493764096:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760753.956597:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24981504, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:2.0:1789760753.956607:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.956628:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:2.0:1789760753.956632:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.956638:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.956651:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760753.989314:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973418 is committed 00002000:00010000:2.0:1789760753.989485:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24981504 -> 24985600 00010000:00010000:2.0:1789760753.989498:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.989511:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.989519:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.989526:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760753.989530:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.989539:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.989558:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.989566:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.989652:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493764096:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33104us (33857us total) trans 4294973418 rc 0/0 00000100:00100000:2.0:1789760753.989663:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6129 00000100:00100000:2.0:1789760753.989671:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764254517588 time 764297568788 state 0x2 ref 0 00000100:00100000:0.0:1789760753.995706:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760753.998090:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493764224 00000100:00100000:2.0:1789760753.998103:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493764224 00000100:00100000:2.0:1789760753.998337:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764254517588 time 764297568788 state 0x0 ref 1 00000100:00100000:2.0:1789760753.998353:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764297829204 time 764297568788 state 0x1 ref 1 00000100:00100000:2.0:1789760753.998392:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764297829204 time 764339419665 state 0x0 ref 1 00000100:00100000:2.0:1789760753.998407:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493764224:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760753.998463:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24985600, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:2.0:1789760753.998472:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cf6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.998488:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:2.0:1789760753.998492:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.998498:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cf6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.998518:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cf6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.042334:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc33eb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519255296:0@lo:400:kworker.0 00000100:00100000:3.0:1789760754.042409:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760754.042479:0:3306:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1789760754.042485:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc33ebb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3306:1876699519255168:0@lo:400:kworker.0 00000100:00100000:3.0:1789760754.042512:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760754.042530:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc33e9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519255680:0@lo:400:kworker.0 00000100:00100000:3.0:1789760754.042550:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760754.042551:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc33eb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3303:1876699519255424:0@lo:400:kworker.0 00000100:00100000:3.0:1789760754.042578:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519255296 00000100:00100000:1.0:1789760754.042580:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760754.042585:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519255296 00000100:00100000:1.0:1789760754.042596:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1789760754.042601:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc33e9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3303:1876699519255040:0@lo:400:kworker.0 00000100:00100000:3.0:1789760754.042624:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519255680 00000100:00100000:1.0:1789760754.042625:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760754.042627:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519255680 00000100:00100000:1.0:1789760754.042645:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc33eaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3303:1876699519255552:0@lo:400:kworker.0 00000100:00100000:3.0:1789760754.042656:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 178 00000100:00100000:1.0:1789760754.042663:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760754.042664:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519255296:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760754.042688:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519255168 00000100:00100000:1.0:1789760754.042691:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519255168 00000100:00100000:3.0:1789760754.042710:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519255296:12345-0@lo:400:kworker.0 Request processed in 48us (300us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760754.042716:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519255424 00000100:00100000:3.0:1789760754.042717:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 178 00000100:00100000:1.0:1789760754.042718:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519255424 00000100:00100000:3.0:1789760754.042731:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 179 00000100:00100000:1.0:1789760754.042735:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519255552 00000100:00100000:1.0:1789760754.042737:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519255552 00000100:00100000:3.0:1789760754.042737:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519255680:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760754.042751:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 494 00000100:00100000:1.0:1789760754.042757:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3306:x1876699519255168:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760754.042762:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519255680:12345-0@lo:400:kworker.0 Request processed in 26us (211us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760754.042767:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 179 00000100:00100000:1.0:1789760754.042785:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ea680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3306:x1876699519255168:12345-0@lo:400:kworker.0 Request processed in 28us (272us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760754.042785:0:9460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 495 00000100:00100000:1.0:1789760754.042790:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 494 00000100:00100000:3.0:1789760754.042791:0:9460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9621018a1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876699519255424:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760754.042802:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 496 00000100:00100000:1.0:1789760754.042808:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9621018a0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876699519255552:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760754.042822:0:9460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9621018a1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876699519255424:12345-0@lo:400:kworker.0 Request processed in 32us (241us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760754.042828:0:9460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 495 00000100:00100000:1.0:1789760754.042829:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9621018a0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876699519255552:12345-0@lo:400:kworker.0 Request processed in 22us (166us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760754.042835:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 496 00000100:00100000:3.0:1789760754.042850:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519255040 00000100:00100000:3.0:1789760754.042854:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519255040 00000100:00100000:3.0:1789760754.042885:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 404 00000100:00100000:3.0:1789760754.042891:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d4b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3303:x1876699519255040:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760754.042919:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d4b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3303:x1876699519255040:12345-0@lo:400:kworker.0 Request processed in 29us (293us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760754.042927:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 404 00000100:00080000:3.0:1789760754.042951:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789760754.042955:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789760754.043006:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc33e9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3303:1876699519255040:0@lo:400:kworker.0 00000100:00100000:3.0:1789760754.043020:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc33eaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3303:1876699519255552:0@lo:400:kworker.0 00000100:00100000:3.0:1789760754.043029:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc33eb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3303:1876699519255424:0@lo:400:kworker.0 00000100:00100000:3.0:1789760754.043057:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc33ebb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3306:1876699519255168:0@lo:400:kworker.0 00000100:00100000:3.0:1789760754.043065:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc33e9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519255680:0@lo:400:kworker.0 00000100:00100000:3.0:1789760754.043073:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc33eb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519255296:0@lo:400:kworker.0 00000001:00080000:1.0:1789760754.058684:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973419 is committed 00002000:00010000:0.0:1789760754.058786:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24985600 -> 24989696 00010000:00010000:0.0:1789760754.058800:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cf6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.058812:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cf6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.058821:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cf6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.058827:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760754.058831:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cf6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.058839:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cf6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.058861:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cf6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.058868:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05cf6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760754.058956:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493764224:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60548us (63255us total) trans 4294973419 rc 0/0 00000100:00100000:0.0:1789760754.059426:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6130 00000100:00100000:0.0:1789760754.059542:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764297829204 time 764339419665 state 0x2 ref 0 00000100:00100000:0.0:1789760754.066737:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760754.066888:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493764352 00000100:00100000:1.0:1789760754.066899:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493764352 00000100:00100000:1.0:1789760754.066953:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764297829204 time 764339419665 state 0x0 ref 1 00000100:00100000:1.0:1789760754.067009:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764339680081 time 764339419665 state 0x1 ref 1 00000100:00100000:1.0:1789760754.067588:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764339680081 time 764408620403 state 0x0 ref 1 00000100:00100000:1.0:1789760754.067607:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7aa7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493764352:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760754.067665:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24989696, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:1.0:1789760754.067677:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05cfd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.067693:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:1.0:1789760754.067697:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.067704:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05cfd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.067718:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05cfd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760754.119599:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973420 is committed 00002000:00010000:3.0:1789760754.119901:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24989696 -> 24993792 00010000:00010000:3.0:1789760754.119916:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05cfd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.119928:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05cfd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.119936:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05cfd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.119943:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760754.119947:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05cfd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.120047:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05cfd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.120068:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05cfd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.120076:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05cfd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.120168:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7aa7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493764352:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52562us (53436us total) trans 4294973420 rc 0/0 00000100:00100000:3.0:1789760754.120178:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6131 00000100:00100000:3.0:1789760754.120189:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764339680081 time 764408620403 state 0x2 ref 0 00000100:00100000:3.0:1789760754.132631:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760754.133179:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493764480 00000100:00100000:0.0:1789760754.133190:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493764480 00000100:00100000:0.0:1789760754.133244:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764339680081 time 764408620403 state 0x0 ref 1 00000100:00100000:0.0:1789760754.133259:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764408880819 time 764408620403 state 0x1 ref 1 00000100:00100000:0.0:1789760754.133293:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764408880819 time 764474328696 state 0x0 ref 1 00000100:00100000:0.0:1789760754.133307:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493764480:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760754.133357:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24993792, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:0.0:1789760754.133367:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05d04 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.133400:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760754.133405:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760754.133411:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05d04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.133425:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05d04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.174595:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973421 is committed 00002000:00010000:1.0:1789760754.174703:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24993792 -> 24997888 00010000:00010000:1.0:1789760754.174718:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05d04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.174731:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05d04 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.174740:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05d04 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.174747:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760754.174752:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05d04 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.174760:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05d04 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.174779:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05d04 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.174787:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05d04 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.174884:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493764480:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41574us (42257us total) trans 4294973421 rc 0/0 00000100:00100000:1.0:1789760754.174896:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6132 00000100:00100000:1.0:1789760754.174906:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764408880819 time 764474328696 state 0x2 ref 0 00000100:00100000:0.0:1789760754.180413:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760754.180554:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493764608 00000100:00100000:2.0:1789760754.180564:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493764608 00000100:00100000:2.0:1789760754.180619:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764408880819 time 764474328696 state 0x0 ref 1 00000100:00100000:2.0:1789760754.180634:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764474589112 time 764474328696 state 0x1 ref 1 00000100:00100000:2.0:1789760754.180663:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764474589112 time 764521699124 state 0x0 ref 1 00000100:00100000:2.0:1789760754.180676:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493764608:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760754.180724:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 24997888, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:2.0:1789760754.180733:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05d0b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.180748:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:2.0:1789760754.180752:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.180758:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05d0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.180771:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05d0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.213831:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973422 is committed 00002000:00010000:3.0:1789760754.213915:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 24997888 -> 25001984 00010000:00010000:3.0:1789760754.213930:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05d0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.213942:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05d0b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.213951:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05d0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.214024:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760754.214029:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05d0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.214038:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05d0b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.214057:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05d0b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.214063:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05d0b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.214185:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493764608:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33507us (33778us total) trans 4294973422 rc 0/0 00000100:00100000:3.0:1789760754.214196:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6133 00000100:00100000:3.0:1789760754.214206:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764474589112 time 764521699124 state 0x2 ref 0 00000100:00100000:0.0:1789760754.218157:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760754.218259:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493764736 00000100:00100000:1.0:1789760754.218269:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493764736 00000100:00100000:1.0:1789760754.218321:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764474589112 time 764521699124 state 0x0 ref 1 00000100:00100000:1.0:1789760754.218337:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764521959540 time 764521699124 state 0x1 ref 1 00000100:00100000:1.0:1789760754.218401:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764521959540 time 764559436331 state 0x0 ref 1 00000100:00100000:1.0:1789760754.218414:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3662680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493764736:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760754.218474:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25001984, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:1.0:1789760754.218482:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05d12 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.218496:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:1.0:1789760754.218501:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.218507:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05d12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.218520:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05d12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.236550:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760754.254766:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25001984 -> 25006080 00010000:00010000:3.0:1789760754.254795:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05d12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.254809:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05d12 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.254823:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05d12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.254831:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760754.254835:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05d12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.254844:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05d12 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.254873:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05d12 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.254882:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e05d12 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.255010:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3662680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493764736:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36593us (36857us total) trans 4294973423 rc 0/0 00000100:00100000:3.0:1789760754.255020:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6134 00000100:00100000:3.0:1789760754.255029:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764521959540 time 764559436331 state 0x2 ref 0 00000001:00080000:0.0:1789760754.256380:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973423 is committed 00000100:00100000:1.0:1789760754.258091:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760754.258194:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493764864 00000100:00100000:3.0:1789760754.258204:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493764864 00000100:00100000:3.0:1789760754.258253:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764521959540 time 764559436331 state 0x0 ref 1 00000100:00100000:3.0:1789760754.258266:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764559696747 time 764559436331 state 0x1 ref 1 00000100:00100000:3.0:1789760754.258290:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764559696747 time 764599326893 state 0x0 ref 1 00000100:00100000:3.0:1789760754.258301:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8794700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493764864:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760754.258348:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25006080, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:3.0:1789760754.258355:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05d19 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.258368:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760754.258372:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.258379:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05d19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.258402:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05d19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760754.294546:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973424 is committed 00002000:00010000:3.0:1789760754.294636:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25006080 -> 25010176 00010000:00010000:3.0:1789760754.294650:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05d19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.294661:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05d19 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.294669:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05d19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.294676:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760754.294681:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05d19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.294690:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05d19 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.294708:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05d19 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.294715:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05d19 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.295121:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8794700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493764864:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36816us (37033us total) trans 4294973424 rc 0/0 00000100:00100000:3.0:1789760754.295131:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6135 00000100:00100000:3.0:1789760754.295140:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764559696747 time 764599326893 state 0x2 ref 0 00000100:00100000:1.0:1789760754.298671:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760754.299368:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493764992 00000100:00100000:3.0:1789760754.299378:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493764992 00000100:00100000:3.0:1789760754.299491:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764559696747 time 764599326893 state 0x0 ref 1 00000100:00100000:3.0:1789760754.299508:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764599587309 time 764599326893 state 0x1 ref 1 00000100:00100000:3.0:1789760754.299570:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764599587309 time 764640606025 state 0x0 ref 1 00000100:00100000:3.0:1789760754.299584:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8796a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493764992:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760754.299633:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25010176, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:3.0:1789760754.299642:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d20 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.299656:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760754.299661:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.299666:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.299679:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.337579:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973425 is committed 00002000:00010000:3.0:1789760754.337679:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25010176 -> 25014272 00010000:00010000:3.0:1789760754.337695:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.337707:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d20 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.337715:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.337723:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760754.337728:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.337736:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d20 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.337759:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d20 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.337766:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d20 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.338039:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8796a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493764992:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38452us (39372us total) trans 4294973425 rc 0/0 00000100:00100000:3.0:1789760754.338050:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6136 00000100:00100000:3.0:1789760754.338059:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764599587309 time 764640606025 state 0x2 ref 0 00000100:00100000:0.0:1789760754.341211:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760754.341291:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493765120 00000100:00100000:3.0:1789760754.341299:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493765120 00000100:00100000:3.0:1789760754.341351:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764599587309 time 764640606025 state 0x0 ref 1 00000100:00100000:3.0:1789760754.341366:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764640866441 time 764640606025 state 0x1 ref 1 00000100:00100000:3.0:1789760754.341408:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764640866441 time 764682443653 state 0x0 ref 1 00000100:00100000:3.0:1789760754.341420:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3662300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493765120:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760754.341479:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25014272, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:3.0:1789760754.341487:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05d27 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.341501:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760754.341506:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.341512:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05d27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.341524:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05d27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760754.400357:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973426 is committed 00002000:00010000:2.0:1789760754.400593:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25014272 -> 25018368 00010000:00010000:2.0:1789760754.400610:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05d27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.400622:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05d27 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.400630:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05d27 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.400638:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760754.400643:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05d27 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.400652:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05d27 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.400671:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05d27 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.400680:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e05d27 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.400768:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3662300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493765120:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59345us (59563us total) trans 4294973426 rc 0/0 00000100:00100000:2.0:1789760754.400778:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6137 00000100:00100000:2.0:1789760754.400787:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764640866441 time 764682443653 state 0x2 ref 0 00000100:00100000:0.0:1789760754.404929:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760754.405192:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493765248 00000100:00100000:1.0:1789760754.405201:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493765248 00000100:00100000:1.0:1789760754.405257:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764640866441 time 764682443653 state 0x0 ref 1 00000100:00100000:1.0:1789760754.405274:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764682704069 time 764682443653 state 0x1 ref 1 00000100:00100000:1.0:1789760754.405307:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764682704069 time 764746342370 state 0x0 ref 1 00000100:00100000:1.0:1789760754.405320:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3661500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493765248:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760754.405368:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25018368, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:1.0:1789760754.405377:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05d2e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.405409:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:1.0:1789760754.405413:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.405419:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05d2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.405433:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05d2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760754.439215:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25018368 -> 25022464 00010000:00010000:3.0:1789760754.439235:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05d2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.439248:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05d2e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.439257:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05d2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.439265:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760754.439269:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05d2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.439277:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05d2e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.439299:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05d2e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.439312:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05d2e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.439560:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3661500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493765248:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34236us (34635us total) trans 4294973427 rc 0/0 00000100:00100000:3.0:1789760754.439572:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6138 00000100:00100000:3.0:1789760754.439581:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764682704069 time 764746342370 state 0x2 ref 0 00000001:00080000:0.0:1789760754.441240:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973427 is committed 00000100:00100000:1.0:1789760754.444526:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760754.444660:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493765376 00000100:00100000:0.0:1789760754.444669:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493765376 00000100:00100000:0.0:1789760754.444724:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764682704069 time 764746342370 state 0x0 ref 1 00000100:00100000:0.0:1789760754.444740:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764746602786 time 764746342370 state 0x1 ref 1 00000100:00100000:0.0:1789760754.444772:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764746602786 time 764785807875 state 0x0 ref 1 00000100:00100000:0.0:1789760754.444784:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493765376:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760754.444832:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25022464, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:0.0:1789760754.444840:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05d35 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.444855:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760754.444860:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760754.444865:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05d35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.444879:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05d35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.480250:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973428 is committed 00002000:00010000:3.0:1789760754.480402:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25022464 -> 25026560 00010000:00010000:3.0:1789760754.480418:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05d35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.480431:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05d35 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.480448:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05d35 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.480456:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760754.480461:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05d35 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.480471:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05d35 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.480500:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05d35 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.480505:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05d35 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.480595:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493765376:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35808us (36073us total) trans 4294973428 rc 0/0 00000100:00100000:3.0:1789760754.480606:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6139 00000100:00100000:3.0:1789760754.480615:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764746602786 time 764785807875 state 0x2 ref 0 00000100:00100000:1.0:1789760754.487881:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760754.489457:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493765504 00000100:00100000:3.0:1789760754.489471:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493765504 00000100:00100000:3.0:1789760754.489543:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764746602786 time 764785807875 state 0x0 ref 1 00000100:00100000:3.0:1789760754.489561:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764786068291 time 764785807875 state 0x1 ref 1 00000100:00100000:3.0:1789760754.489599:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764786068291 time 764830634415 state 0x0 ref 1 00000100:00100000:3.0:1789760754.489613:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493765504:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760754.489663:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25026560, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:3.0:1789760754.489673:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d3c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.489689:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760754.489695:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.489701:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.489716:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760754.544695:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973429 is committed 00002000:00010000:3.0:1789760754.544794:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25026560 -> 25030656 00010000:00010000:3.0:1789760754.544810:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.545558:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d3c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.545567:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d3c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.545583:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760754.545588:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d3c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.545598:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d3c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.545618:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d3c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.545626:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d3c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.545720:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493765504:12345-192.168.203.60@tcp:4:dd.0 Request processed in 56105us (57843us total) trans 4294973429 rc 0/0 00000100:00100000:3.0:1789760754.545731:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6140 00000100:00100000:3.0:1789760754.545741:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764786068291 time 764830634415 state 0x2 ref 0 00000100:00100000:0.0:1789760754.548744:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760754.548837:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493765632 00000100:00100000:2.0:1789760754.548847:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493765632 00000100:00100000:2.0:1789760754.548920:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764786068291 time 764830634415 state 0x0 ref 1 00000100:00100000:2.0:1789760754.548934:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764830894831 time 764830634415 state 0x1 ref 1 00000100:00100000:2.0:1789760754.549004:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764830894831 time 764890038847 state 0x0 ref 1 00000100:00100000:2.0:1789760754.549018:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493765632:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760754.549069:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25030656, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:2.0:1789760754.549078:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05d43 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.549094:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:2.0:1789760754.549099:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.549106:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05d43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.549120:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05d43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.556324:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760754.556348:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc452d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519255808:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760754.556432:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760754.557423:0:6691:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519255808 00000100:00100000:1.0:1789760754.557432:0:6691:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519255808 00000100:00100000:1.0:1789760754.557500:0:6691:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 205 00000100:00100000:1.0:1789760754.557510:0:6691:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519255808:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760754.557547:0:6691:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760754.557603:0:6691:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519255808:12345-0@lo:13:osp-pre-1-0.0 Request processed in 95us (1171us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760754.557612:0:6691:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 205 00000100:00100000:1.0:1789760754.557661:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc452d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519255808:0@lo:13:osp-pre-1-0.0 00000001:00080000:0.0:1789760754.590173:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973430 is committed 00002000:00010000:3.0:1789760754.590564:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25030656 -> 25034752 00010000:00010000:3.0:1789760754.590579:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05d43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.590593:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05d43 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.590603:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05d43 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.590610:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760754.590615:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05d43 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.590623:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05d43 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.590804:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05d43 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.590819:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05d43 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.591004:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493765632:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41939us (42223us total) trans 4294973430 rc 0/0 00000100:00100000:3.0:1789760754.591017:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6141 00000100:00100000:3.0:1789760754.591027:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764830894831 time 764890038847 state 0x2 ref 0 00000100:00100000:1.0:1789760754.594864:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760754.595700:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493765760 00000100:00100000:3.0:1789760754.595713:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493765760 00000100:00100000:3.0:1789760754.595770:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764830894831 time 764890038847 state 0x0 ref 1 00000100:00100000:3.0:1789760754.595790:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764890299263 time 764890038847 state 0x1 ref 1 00000100:00100000:3.0:1789760754.595820:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764890299263 time 764936855506 state 0x0 ref 1 00000100:00100000:3.0:1789760754.595833:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493765760:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760754.596066:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25034752, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xc007 00010000:00010000:3.0:1789760754.596073:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d4a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.596086:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760754.596089:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.596093:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.596107:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760754.637929:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25034752 -> 25038848 00002000:00010000:1.0:1789760754.637948:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760753 -> 1789760754 00002000:00010000:1.0:1789760754.637952:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760753 -> 1789760754 00010000:00010000:1.0:1789760754.637990:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.638003:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d4a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.638011:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.638019:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760754.638024:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.638039:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d4a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.638058:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d4a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.638066:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e05d4a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.638148:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493765760:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42313us (43289us total) trans 4294973431 rc 0/0 00000100:00100000:1.0:1789760754.638159:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6142 00000100:00100000:1.0:1789760754.638169:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764890299263 time 764936855506 state 0x2 ref 0 00000001:00080000:2.0:1789760754.638889:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973431 is committed 00000100:00100000:1.0:1789760754.642549:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760754.642663:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493765888 00000100:00100000:3.0:1789760754.642673:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493765888 00000100:00100000:3.0:1789760754.642727:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764890299263 time 764936855506 state 0x0 ref 1 00000100:00100000:3.0:1789760754.642742:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764937115922 time 764936855506 state 0x1 ref 1 00000100:00100000:3.0:1789760754.642775:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764937115922 time 764983810178 state 0x0 ref 1 00000100:00100000:3.0:1789760754.642787:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493765888:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760754.642833:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25038848, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:3.0:1789760754.642841:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05d51 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.642855:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760754.642860:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.642866:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05d51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.642891:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05d51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760754.687837:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973432 is committed 00002000:00010000:3.0:1789760754.688155:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25038848 -> 25042944 00010000:00010000:3.0:1789760754.688171:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05d51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.688185:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05d51 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.688195:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05d51 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.688202:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760754.688208:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05d51 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.688218:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05d51 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.688240:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05d51 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.688248:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05d51 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.688397:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493765888:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45595us (45841us total) trans 4294973432 rc 0/0 00000100:00100000:3.0:1789760754.688407:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6143 00000100:00100000:3.0:1789760754.688418:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764937115922 time 764983810178 state 0x2 ref 0 00000100:00100000:3.0:1789760754.696726:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760754.696897:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493766016 00000100:00100000:1.0:1789760754.696908:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493766016 00000100:00100000:1.0:1789760754.697017:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764937115922 time 764983810178 state 0x0 ref 1 00000100:00100000:1.0:1789760754.697032:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764984070594 time 764983810178 state 0x1 ref 1 00000100:00100000:1.0:1789760754.697065:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764984070594 time 765038101085 state 0x0 ref 1 00000100:00100000:1.0:1789760754.697079:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493766016:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760754.697132:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25042944, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:1.0:1789760754.697140:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05d58 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.697157:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760754.697161:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.697167:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05d58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.697182:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05d58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760754.742105:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25042944 -> 25047040 00010000:00010000:1.0:1789760754.742123:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05d58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.742137:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05d58 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.742147:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05d58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.742155:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760754.742160:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05d58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.742170:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05d58 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.742192:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05d58 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.742200:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05d58 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.742305:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493766016:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45224us (45586us total) trans 4294973433 rc 0/0 00000100:00100000:1.0:1789760754.742316:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6144 00000100:00100000:1.0:1789760754.742327:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764984070594 time 765038101085 state 0x2 ref 0 00000001:00080000:1.0:1789760754.742367:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973433 is committed 00000100:00100000:1.0:1789760754.748906:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760754.749110:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493766144 00000100:00100000:2.0:1789760754.749119:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493766144 00000100:00100000:2.0:1789760754.749173:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 764984070594 time 765038101085 state 0x0 ref 1 00000100:00100000:2.0:1789760754.749187:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765038361501 time 765038101085 state 0x1 ref 1 00000100:00100000:2.0:1789760754.749256:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765038361501 time 765090290813 state 0x0 ref 1 00000100:00100000:2.0:1789760754.749269:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493766144:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760754.749314:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25047040, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:2.0:1789760754.749323:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05d5f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.749337:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760754.749342:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.749348:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05d5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.749361:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05d5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760754.775697:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973434 is committed 00002000:00010000:2.0:1789760754.775811:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25047040 -> 25051136 00010000:00010000:2.0:1789760754.775828:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05d5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.775842:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05d5f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.775852:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05d5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.775859:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760754.775865:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05d5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.775874:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05d5f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.775894:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05d5f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.775900:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e05d5f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.777245:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493766144:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27967us (28338us total) trans 4294973434 rc 0/0 00000100:00100000:2.0:1789760754.777258:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6145 00000100:00100000:2.0:1789760754.777269:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765038361501 time 765090290813 state 0x2 ref 0 00000100:00100000:2.0:1789760754.783134:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760754.783193:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493766272 00000100:00100000:3.0:1789760754.783203:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493766272 00000100:00100000:3.0:1789760754.783255:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765038361501 time 765090290813 state 0x0 ref 1 00000100:00100000:3.0:1789760754.783270:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765090551229 time 765090290813 state 0x1 ref 1 00000100:00100000:3.0:1789760754.783301:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765090551229 time 765124337265 state 0x0 ref 1 00000100:00100000:3.0:1789760754.783312:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493766272:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760754.783359:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25051136, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:3.0:1789760754.783367:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05d66 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.783394:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:3.0:1789760754.783399:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.783405:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05d66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.783418:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05d66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760754.819181:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973435 is committed 00002000:00010000:1.0:1789760754.819296:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25051136 -> 25055232 00010000:00010000:1.0:1789760754.819310:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05d66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.819318:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05d66 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.819324:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05d66 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.819328:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760754.819332:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05d66 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.819338:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05d66 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.819354:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05d66 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.819358:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e05d66 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.819545:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493766272:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36229us (36425us total) trans 4294973435 rc 0/0 00000100:00100000:1.0:1789760754.819553:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6146 00000100:00100000:1.0:1789760754.819561:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765090551229 time 765124337265 state 0x2 ref 0 00000100:00100000:0.0:1789760754.823804:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760754.824177:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493766400 00000100:00100000:3.0:1789760754.824188:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493766400 00000100:00100000:3.0:1789760754.824248:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765090551229 time 765124337265 state 0x0 ref 1 00000100:00100000:3.0:1789760754.824265:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765124597681 time 765124337265 state 0x1 ref 1 00000100:00100000:3.0:1789760754.824297:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765124597681 time 765165332708 state 0x0 ref 1 00000100:00100000:3.0:1789760754.824309:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493766400:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760754.824355:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25055232, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:3.0:1789760754.824364:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05d6d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.824390:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:3.0:1789760754.824395:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.824401:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05d6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.824526:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05d6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.871101:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973436 is committed 00002000:00010000:1.0:1789760754.871239:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25055232 -> 25059328 00010000:00010000:1.0:1789760754.871257:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05d6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.871273:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05d6d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.871282:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05d6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.871289:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760754.871296:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05d6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.871306:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05d6d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.871329:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05d6d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.871339:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e05d6d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.871447:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493766400:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47128us (47640us total) trans 4294973436 rc 0/0 00000100:00100000:1.0:1789760754.871459:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6147 00000100:00100000:1.0:1789760754.871539:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765124597681 time 765165332708 state 0x2 ref 0 00000100:00100000:2.0:1789760754.875835:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760754.876020:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493766528 00000100:00100000:1.0:1789760754.876032:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493766528 00000100:00100000:1.0:1789760754.876089:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765124597681 time 765165332708 state 0x0 ref 1 00000100:00100000:1.0:1789760754.876105:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765165593124 time 765165332708 state 0x1 ref 1 00000100:00100000:1.0:1789760754.876138:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765165593124 time 765217173472 state 0x0 ref 1 00000100:00100000:1.0:1789760754.876150:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493766528:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760754.876327:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25059328, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:1.0:1789760754.876337:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05d74 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.876352:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760754.876357:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.876362:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05d74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.876389:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05d74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760754.904288:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25059328 -> 25063424 00010000:00010000:0.0:1789760754.904307:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05d74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.904319:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05d74 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.904328:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05d74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.904350:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760754.904354:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05d74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.904363:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05d74 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.904394:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05d74 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.904402:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05d74 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760754.904497:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493766528:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28344us (28668us total) trans 4294973437 rc 0/0 00000100:00100000:0.0:1789760754.904509:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6148 00000100:00100000:0.0:1789760754.904518:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765165593124 time 765217173472 state 0x2 ref 0 00000001:00080000:0.0:1789760754.904559:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973437 is committed 00000100:00100000:2.0:1789760754.911792:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760754.911929:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493766656 00000100:00100000:3.0:1789760754.911940:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493766656 00000100:00100000:3.0:1789760754.912040:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765165593124 time 765217173472 state 0x0 ref 1 00000100:00100000:3.0:1789760754.912056:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765217433888 time 765217173472 state 0x1 ref 1 00000100:00100000:3.0:1789760754.912089:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765217433888 time 765253124405 state 0x0 ref 1 00000100:00100000:3.0:1789760754.912102:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493766656:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760754.912153:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25063424, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:3.0:1789760754.912162:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05d7b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.912179:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:3.0:1789760754.912185:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.912191:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05d7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.912204:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05d7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760754.942659:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25063424 -> 25067520 00010000:00010000:0.0:1789760754.942679:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05d7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.942693:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05d7b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.942706:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05d7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.942713:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760754.942717:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05d7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.942724:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05d7b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.942743:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05d7b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.942750:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05d7b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760754.942848:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493766656:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30743us (31060us total) trans 4294973438 rc 0/0 00000100:00100000:0.0:1789760754.942858:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6149 00000100:00100000:0.0:1789760754.942868:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765217433888 time 765253124405 state 0x2 ref 0 00000001:00080000:1.0:1789760754.943847:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973438 is committed 00000100:00100000:1.0:1789760754.946519:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760754.946604:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493766784 00000100:00100000:3.0:1789760754.946614:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493766784 00000100:00100000:3.0:1789760754.946666:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765217433888 time 765253124405 state 0x0 ref 1 00000100:00100000:3.0:1789760754.946681:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765253384821 time 765253124405 state 0x1 ref 1 00000100:00100000:3.0:1789760754.946744:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765253384821 time 765287779469 state 0x0 ref 1 00000100:00100000:3.0:1789760754.946757:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493766784:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760754.946803:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25067520, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:3.0:1789760754.946811:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05d82 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.946827:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:3.0:1789760754.946833:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.946839:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05d82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.946853:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05d82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760754.984796:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973439 is committed 00002000:00010000:0.0:1789760754.984945:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25067520 -> 25071616 00010000:00010000:0.0:1789760754.984976:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05d82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.984996:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05d82 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.985006:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05d82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.985014:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760754.985019:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05d82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.985030:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05d82 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.985050:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05d82 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.985058:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05d82 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760754.985149:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493766784:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38389us (38635us total) trans 4294973439 rc 0/0 00000100:00100000:0.0:1789760754.985160:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6150 00000100:00100000:0.0:1789760754.986671:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765253384821 time 765287779469 state 0x2 ref 0 00000100:00100000:2.0:1789760754.989134:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760754.989312:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493766912 00000100:00100000:1.0:1789760754.989323:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493766912 00000100:00100000:1.0:1789760754.989392:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765253384821 time 765287779469 state 0x0 ref 1 00000100:00100000:1.0:1789760754.989408:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765288039885 time 765287779469 state 0x1 ref 1 00000100:00100000:1.0:1789760754.989475:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765288039885 time 765330509270 state 0x0 ref 1 00000100:00100000:1.0:1789760754.989488:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493766912:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760754.989537:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25071616, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:1.0:1789760754.989547:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05d89 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.989563:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760754.989569:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.989576:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05d89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.989590:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05d89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.051205:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973440 is committed 00002000:00010000:1.0:1789760755.051342:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25071616 -> 25075712 00010000:00010000:1.0:1789760755.051358:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05d89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.051367:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05d89 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.051373:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05d89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.051392:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760755.051399:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05d89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.051408:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05d89 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.051427:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05d89 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.051496:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05d89 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.051586:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493766912:12345-192.168.203.60@tcp:4:dd.0 Request processed in 62096us (62455us total) trans 4294973440 rc 0/0 00000100:00100000:1.0:1789760755.051597:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6151 00000100:00100000:1.0:1789760755.051606:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765288039885 time 765330509270 state 0x2 ref 0 00000100:00100000:2.0:1789760755.054528:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760755.054618:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493767040 00000100:00100000:0.0:1789760755.054627:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493767040 00000100:00100000:0.0:1789760755.054730:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765288039885 time 765330509270 state 0x0 ref 1 00000100:00100000:0.0:1789760755.054746:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765330769686 time 765330509270 state 0x1 ref 1 00000100:00100000:0.0:1789760755.054781:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765330769686 time 765395816359 state 0x0 ref 1 00000100:00100000:0.0:1789760755.054794:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493767040:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760755.054841:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25075712, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:0.0:1789760755.054850:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05d90 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.054865:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760755.054869:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.054875:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05d90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.054889:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05d90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760755.055927:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760755.085585:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973441 is committed 00002000:00010000:1.0:1789760755.085670:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25075712 -> 25079808 00010000:00010000:1.0:1789760755.085686:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05d90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.085699:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05d90 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.085707:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05d90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.085713:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760755.085718:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05d90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.085726:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05d90 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.085749:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05d90 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.085756:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05d90 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.085892:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493767040:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31097us (31369us total) trans 4294973441 rc 0/0 00000100:00100000:1.0:1789760755.085903:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6152 00000100:00100000:1.0:1789760755.085913:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765330769686 time 765395816359 state 0x2 ref 0 00000100:00100000:0.0:1789760755.092900:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760755.092998:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493767168 00000100:00100000:3.0:1789760755.093008:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493767168 00000100:00100000:3.0:1789760755.093062:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765330769686 time 765395816359 state 0x0 ref 1 00000100:00100000:3.0:1789760755.093078:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765396076775 time 765395816359 state 0x1 ref 1 00000100:00100000:3.0:1789760755.093113:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765396076775 time 765434148560 state 0x0 ref 1 00000100:00100000:3.0:1789760755.093126:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493767168:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760755.093176:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25079808, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:3.0:1789760755.093186:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05d97 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.093202:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:3.0:1789760755.093206:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.093212:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05d97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.093231:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05d97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760755.136712:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973442 is committed 00002000:00010000:3.0:1789760755.136791:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25079808 -> 25083904 00010000:00010000:3.0:1789760755.136806:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05d97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.136818:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05d97 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.136827:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05d97 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.136835:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760755.136839:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05d97 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.136849:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05d97 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.136868:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05d97 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.136875:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05d97 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.137046:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493767168:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43917us (44150us total) trans 4294973442 rc 0/0 00000100:00100000:3.0:1789760755.137056:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6153 00000100:00100000:3.0:1789760755.137065:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765396076775 time 765434148560 state 0x2 ref 0 00000100:00100000:3.0:1789760755.140878:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760755.141495:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493767296 00000100:00100000:1.0:1789760755.141505:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493767296 00000100:00100000:0.0:1789760755.141696:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765396076775 time 765434148560 state 0x0 ref 1 00000100:00100000:0.0:1789760755.141708:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765434408976 time 765434148560 state 0x1 ref 1 00000100:00100000:0.0:1789760755.141737:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765434408976 time 765482772731 state 0x0 ref 1 00000100:00100000:0.0:1789760755.141754:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493767296:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760755.141801:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25083904, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:0.0:1789760755.141809:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05d9e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.141822:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760755.141825:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.141830:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05d9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.141840:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05d9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.193077:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973443 is committed 00002000:00010000:2.0:1789760755.193162:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25083904 -> 25088000 00010000:00010000:2.0:1789760755.193177:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05d9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.193191:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05d9e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.193200:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05d9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.193209:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760755.193214:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05d9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.193224:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05d9e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.193243:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05d9e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.193252:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05d9e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.193449:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493767296:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51692us (52574us total) trans 4294973443 rc 0/0 00000100:00100000:2.0:1789760755.193460:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6154 00000100:00100000:2.0:1789760755.193470:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765434408976 time 765482772731 state 0x2 ref 0 00000100:00100000:2.0:1789760755.197949:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760755.198143:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493767424 00000100:00100000:1.0:1789760755.198157:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493767424 00000100:00100000:1.0:1789760755.198222:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765434408976 time 765482772731 state 0x0 ref 1 00000100:00100000:1.0:1789760755.198239:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765483033147 time 765482772731 state 0x1 ref 1 00000100:00100000:1.0:1789760755.198265:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765483033147 time 765539300447 state 0x0 ref 1 00000100:00100000:1.0:1789760755.198278:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493767424:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760755.198335:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25088000, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:1.0:1789760755.198345:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05da5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.198365:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760755.198369:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760755.198386:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05da5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.198400:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05da5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.227404:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973444 is committed 00002000:00010000:1.0:1789760755.227459:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25088000 -> 25092096 00010000:00010000:1.0:1789760755.227474:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05da5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.227486:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05da5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.227495:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05da5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.227502:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760755.227508:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05da5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.227517:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05da5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.227537:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05da5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.227545:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05da5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.227636:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493767424:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29356us (29692us total) trans 4294973444 rc 0/0 00000100:00100000:1.0:1789760755.227646:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6155 00000100:00100000:1.0:1789760755.227656:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765483033147 time 765539300447 state 0x2 ref 0 00000100:00100000:0.0:1789760755.232710:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760755.233005:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493767552 00000100:00100000:1.0:1789760755.233015:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493767552 00000100:00100000:1.0:1789760755.233076:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765483033147 time 765539300447 state 0x0 ref 1 00000100:00100000:1.0:1789760755.233091:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765539560863 time 765539300447 state 0x1 ref 1 00000100:00100000:1.0:1789760755.233779:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765539560863 time 765574811732 state 0x0 ref 1 00000100:00100000:1.0:1789760755.233796:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493767552:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760755.233846:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25092096, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:1.0:1789760755.233855:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05dac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.233870:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760755.233875:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760755.233881:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05dac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.233899:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05dac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.258044:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760755.273265:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973445 is committed 00002000:00010000:1.0:1789760755.273347:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25092096 -> 25096192 00010000:00010000:1.0:1789760755.273363:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05dac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.273384:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05dac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.273393:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05dac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.273399:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760755.273404:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05dac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.273412:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05dac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.273439:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05dac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.273447:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05dac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.273688:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493767552:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39891us (40981us total) trans 4294973445 rc 0/0 00000100:00100000:1.0:1789760755.273700:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6156 00000100:00100000:1.0:1789760755.273709:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765539560863 time 765574811732 state 0x2 ref 0 00000100:00100000:3.0:1789760755.276178:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760755.276293:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493767680 00000100:00100000:0.0:1789760755.276303:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493767680 00000100:00100000:0.0:1789760755.276349:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765539560863 time 765574811732 state 0x0 ref 1 00000100:00100000:0.0:1789760755.276363:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765575072148 time 765574811732 state 0x1 ref 1 00000100:00100000:0.0:1789760755.276399:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765575072148 time 765617434822 state 0x0 ref 1 00000100:00100000:0.0:1789760755.276412:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493767680:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760755.276537:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25096192, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:0.0:1789760755.276546:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05db3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.276560:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760755.276565:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.276570:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05db3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.276584:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05db3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.305741:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973446 is committed 00002000:00010000:1.0:1789760755.306047:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25096192 -> 25100288 00010000:00010000:1.0:1789760755.306060:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05db3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.306072:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05db3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.306080:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05db3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.306087:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760755.306091:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05db3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.306100:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05db3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.306123:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05db3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.306136:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e05db3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.306260:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493767680:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29845us (30085us total) trans 4294973446 rc 0/0 00000100:00100000:1.0:1789760755.306271:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6157 00000100:00100000:1.0:1789760755.306280:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765575072148 time 765617434822 state 0x2 ref 0 00000100:00100000:2.0:1789760755.310774:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760755.310926:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493767808 00000100:00100000:0.0:1789760755.310936:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493767808 00000100:00100000:0.0:1789760755.311098:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765575072148 time 765617434822 state 0x0 ref 1 00000100:00100000:0.0:1789760755.311114:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765617695238 time 765617434822 state 0x1 ref 1 00000100:00100000:0.0:1789760755.311170:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765617695238 time 765652205599 state 0x0 ref 1 00000100:00100000:0.0:1789760755.311183:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493767808:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760755.311233:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25100288, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:0.0:1789760755.311241:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05dba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.311256:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760755.311261:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.311267:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05dba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.311281:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05dba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760755.336027:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973447 is committed 00002000:00010000:0.0:1789760755.336205:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25100288 -> 25104384 00010000:00010000:0.0:1789760755.336220:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05dba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.336232:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05dba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.336241:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05dba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.336248:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760755.336254:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05dba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.336262:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05dba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.336281:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05dba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.336288:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05dba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760755.336422:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493767808:12345-192.168.203.60@tcp:4:dd.0 Request processed in 25237us (25651us total) trans 4294973447 rc 0/0 00000100:00100000:0.0:1789760755.336442:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6158 00000100:00100000:0.0:1789760755.336451:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765617695238 time 765652205599 state 0x2 ref 0 00000100:00100000:0.0:1789760755.339466:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760755.339601:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493767936 00000100:00100000:1.0:1789760755.339611:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493767936 00000100:00100000:1.0:1789760755.339665:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765617695238 time 765652205599 state 0x0 ref 1 00000100:00100000:1.0:1789760755.339679:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765652466015 time 765652205599 state 0x1 ref 1 00000100:00100000:1.0:1789760755.339710:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765652466015 time 765680746034 state 0x0 ref 1 00000100:00100000:1.0:1789760755.339723:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493767936:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760755.339771:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25104384, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:1.0:1789760755.339779:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05dc1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.339794:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760755.339799:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760755.339805:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05dc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.339825:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05dc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760755.392717:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973448 is committed 00002000:00010000:0.0:1789760755.393098:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25104384 -> 25108480 00010000:00010000:0.0:1789760755.393111:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05dc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.393120:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05dc1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.393125:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05dc1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.393130:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760755.393135:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05dc1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.393144:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05dc1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.393161:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05dc1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.393174:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05dc1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760755.393262:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493767936:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53538us (53802us total) trans 4294973448 rc 0/0 00000100:00100000:0.0:1789760755.393271:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6159 00000100:00100000:0.0:1789760755.393278:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765652466015 time 765680746034 state 0x2 ref 0 00000100:00100000:1.0:1789760755.396846:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760755.397000:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493768064 00000100:00100000:0.0:1789760755.397009:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493768064 00000100:00100000:0.0:1789760755.397062:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765652466015 time 765680746034 state 0x0 ref 1 00000100:00100000:0.0:1789760755.397078:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765681006450 time 765680746034 state 0x1 ref 1 00000100:00100000:0.0:1789760755.397114:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765681006450 time 765738149817 state 0x0 ref 1 00000100:00100000:0.0:1789760755.397127:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493768064:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760755.397173:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25108480, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:0.0:1789760755.397182:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05dc8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.397198:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760755.397203:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.397560:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05dc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.397581:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05dc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.437765:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25108480 -> 25112576 00010000:00010000:3.0:1789760755.437786:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05dc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760755.437797:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973449 is committed 00010000:00010000:3.0:1789760755.437799:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05dc8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.437807:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05dc8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.437815:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760755.437820:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05dc8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.437829:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05dc8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.437849:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05dc8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.437866:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e05dc8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.438143:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493768064:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41013us (41305us total) trans 4294973449 rc 0/0 00000100:00100000:3.0:1789760755.438153:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6160 00000100:00100000:3.0:1789760755.438163:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765681006450 time 765738149817 state 0x2 ref 0 00000100:00100000:2.0:1789760755.442205:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760755.442298:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493768192 00000100:00100000:0.0:1789760755.442308:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493768192 00000100:00100000:0.0:1789760755.442363:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765681006450 time 765738149817 state 0x0 ref 1 00000100:00100000:0.0:1789760755.442386:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765738410233 time 765738149817 state 0x1 ref 1 00000100:00100000:0.0:1789760755.442416:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765738410233 time 765783451268 state 0x0 ref 1 00000100:00100000:0.0:1789760755.442487:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493768192:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760755.442541:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25112576, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:0.0:1789760755.442550:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05dcf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.442565:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760755.442569:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.442576:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05dcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.442589:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05dcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.478811:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973450 is committed 00002000:00010000:0.0:1789760755.478954:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25112576 -> 25116672 00010000:00010000:0.0:1789760755.478988:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05dcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.479000:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05dcf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.479009:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05dcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.479016:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760755.479020:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05dcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.479029:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05dcf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.479047:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05dcf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.479055:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05dcf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760755.479185:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493768192:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36753us (36984us total) trans 4294973450 rc 0/0 00000100:00100000:0.0:1789760755.479204:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6161 00000100:00100000:0.0:1789760755.479214:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765738410233 time 765783451268 state 0x2 ref 0 00000100:00100000:0.0:1789760755.483193:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760755.483421:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493768320 00000100:00100000:2.0:1789760755.483438:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493768320 00000100:00100000:2.0:1789760755.483492:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765738410233 time 765783451268 state 0x0 ref 1 00000100:00100000:2.0:1789760755.483508:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765783711684 time 765783451268 state 0x1 ref 1 00000100:00100000:2.0:1789760755.483571:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765783711684 time 765824605914 state 0x0 ref 1 00000100:00100000:2.0:1789760755.483583:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493768320:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760755.483629:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25116672, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:2.0:1789760755.483637:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05dd6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.483651:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760755.483655:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760755.483661:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05dd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.483675:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05dd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760755.515008:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973451 is committed 00002000:00010000:2.0:1789760755.515193:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25116672 -> 25120768 00010000:00010000:2.0:1789760755.515262:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05dd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.515273:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05dd6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.515278:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05dd6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.515282:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760755.515286:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05dd6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.515293:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05dd6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.515309:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05dd6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.515313:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e05dd6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.515500:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493768320:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31913us (32310us total) trans 4294973451 rc 0/0 00000100:00100000:2.0:1789760755.515511:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6162 00000100:00100000:2.0:1789760755.515521:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765783711684 time 765824605914 state 0x2 ref 0 00000100:00100000:1.0:1789760755.518334:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760755.518461:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493768448 00000100:00100000:0.0:1789760755.518470:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493768448 00000100:00100000:0.0:1789760755.518525:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765783711684 time 765824605914 state 0x0 ref 1 00000100:00100000:0.0:1789760755.518539:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765824866330 time 765824605914 state 0x1 ref 1 00000100:00100000:0.0:1789760755.518568:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765824866330 time 765859603288 state 0x0 ref 1 00000100:00100000:0.0:1789760755.518579:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493768448:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760755.518625:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25120768, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:0.0:1789760755.518634:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05ddd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.518649:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760755.518654:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.518659:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05ddd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.518674:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05ddd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760755.555055:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973452 is committed 00002000:00010000:3.0:1789760755.555168:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25120768 -> 25124864 00010000:00010000:3.0:1789760755.555184:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05ddd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.555197:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05ddd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.555206:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05ddd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.555214:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760755.555219:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05ddd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.555235:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05ddd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.555254:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05ddd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.555263:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05ddd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.555355:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493768448:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36772us (37025us total) trans 4294973452 rc 0/0 00000100:00100000:3.0:1789760755.555364:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6163 00000100:00100000:3.0:1789760755.555384:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765824866330 time 765859603288 state 0x2 ref 0 00000100:00100000:1.0:1789760755.559902:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760755.560086:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493768576 00000100:00100000:0.0:1789760755.560096:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493768576 00000100:00100000:0.0:1789760755.560148:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765824866330 time 765859603288 state 0x0 ref 1 00000100:00100000:0.0:1789760755.560160:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765859863704 time 765859603288 state 0x1 ref 1 00000100:00100000:0.0:1789760755.560183:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765859863704 time 765901219465 state 0x0 ref 1 00000100:00100000:0.0:1789760755.560191:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493768576:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760755.560240:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25124864, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:0.0:1789760755.560248:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05de4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.560263:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760755.560268:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.560275:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05de4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.560288:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05de4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760755.608190:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25124864 -> 25128960 00010000:00010000:0.0:1789760755.608208:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05de4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.608226:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05de4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.608240:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05de4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.608248:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760755.608253:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05de4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.608262:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05de4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.608282:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05de4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.608290:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05de4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760755.608369:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493768576:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48172us (48472us total) trans 4294973453 rc 0/0 00000100:00100000:0.0:1789760755.608390:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6164 00000100:00100000:0.0:1789760755.608407:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765859863704 time 765901219465 state 0x2 ref 0 00000001:00080000:2.0:1789760755.608797:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973453 is committed 00000100:00100000:2.0:1789760755.611402:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760755.611488:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493768704 00000100:00100000:1.0:1789760755.611499:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493768704 00000100:00100000:1.0:1789760755.611554:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765859863704 time 765901219465 state 0x0 ref 1 00000100:00100000:1.0:1789760755.611570:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765901479881 time 765901219465 state 0x1 ref 1 00000100:00100000:1.0:1789760755.611602:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765901479881 time 765952637757 state 0x0 ref 1 00000100:00100000:1.0:1789760755.611615:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493768704:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760755.611661:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25128960, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xc007 00010000:00010000:1.0:1789760755.611669:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05deb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.611684:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760755.611689:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760755.611695:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05deb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.611717:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05deb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760755.647690:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973454 is committed 00002000:00010000:1.0:1789760755.647789:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25128960 -> 25133056 00002000:00010000:1.0:1789760755.647793:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760754 -> 1789760755 00002000:00010000:1.0:1789760755.647796:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760754 -> 1789760755 00010000:00010000:1.0:1789760755.647809:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05deb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.647823:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05deb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.647833:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05deb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.647841:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760755.647846:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05deb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.647856:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05deb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.647875:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05deb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.647887:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05deb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.648002:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493768704:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36385us (36612us total) trans 4294973454 rc 0/0 00000100:00100000:1.0:1789760755.648013:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6165 00000100:00100000:1.0:1789760755.648022:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765901479881 time 765952637757 state 0x2 ref 0 00000100:00100000:2.0:1789760755.654355:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760755.654639:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493768832 00000100:00100000:0.0:1789760755.654651:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493768832 00000100:00100000:0.0:1789760755.654706:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765901479881 time 765952637757 state 0x0 ref 1 00000100:00100000:0.0:1789760755.654723:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765952898173 time 765952637757 state 0x1 ref 1 00000100:00100000:0.0:1789760755.654802:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765952898173 time 765995836295 state 0x0 ref 1 00000100:00100000:0.0:1789760755.654816:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493768832:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760755.654866:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25133056, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc007 00010000:00010000:0.0:1789760755.654875:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05df2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.654891:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760755.654895:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.654901:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05df2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.654914:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05df2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.689828:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973455 is committed 00002000:00010000:2.0:1789760755.690104:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25133056 -> 25137152 00010000:00010000:2.0:1789760755.690120:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05df2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.690133:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05df2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.690141:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05df2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.690148:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760755.690153:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05df2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.690162:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05df2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.690183:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05df2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.690191:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e05df2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.690314:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493768832:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35495us (35964us total) trans 4294973455 rc 0/0 00000100:00100000:2.0:1789760755.690326:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6166 00000100:00100000:2.0:1789760755.690336:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765952898173 time 765995836295 state 0x2 ref 0 00000100:00100000:2.0:1789760755.693793:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760755.693869:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493768960 00000100:00100000:3.0:1789760755.693879:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493768960 00000100:00100000:3.0:1789760755.694108:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765952898173 time 765995836295 state 0x0 ref 1 00000100:00100000:3.0:1789760755.694122:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765996096711 time 765995836295 state 0x1 ref 1 00000100:00100000:3.0:1789760755.694180:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765996096711 time 766035215631 state 0x0 ref 1 00000100:00100000:3.0:1789760755.694192:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493768960:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760755.694241:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25137152, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc007 00010000:00010000:3.0:1789760755.694249:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05df9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.694263:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760755.694268:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.694275:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05df9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.694288:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05df9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760755.726664:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25137152 -> 25141248 00010000:00010000:1.0:1789760755.726688:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05df9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.726702:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05df9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.726712:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05df9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.726720:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760755.726724:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05df9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.726734:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05df9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.726761:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05df9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.726769:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05df9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.726861:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493768960:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32665us (33070us total) trans 4294973456 rc 0/0 00000100:00100000:1.0:1789760755.726871:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6167 00000100:00100000:1.0:1789760755.726879:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765996096711 time 766035215631 state 0x2 ref 0 00000001:00080000:3.0:1789760755.727276:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973456 is committed 00000100:00100000:2.0:1789760755.730912:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760755.731146:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493769088 00000100:00100000:3.0:1789760755.731154:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493769088 00000100:00100000:3.0:1789760755.731201:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 765996096711 time 766035215631 state 0x0 ref 1 00000100:00100000:3.0:1789760755.731221:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766035476047 time 766035215631 state 0x1 ref 1 00000100:00100000:3.0:1789760755.731239:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766035476047 time 766072276469 state 0x0 ref 1 00000100:00100000:3.0:1789760755.731248:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493769088:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760755.731297:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25141248, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc007 00010000:00010000:3.0:1789760755.731304:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05e00 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.731320:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760755.731324:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.731330:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05e00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.731341:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05e00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760755.783294:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973457 is committed 00002000:00010000:1.0:1789760755.783615:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25141248 -> 25145344 00010000:00010000:1.0:1789760755.783629:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05e00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.783643:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05e00 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.783653:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05e00 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.783661:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760755.783666:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05e00 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.783682:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05e00 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.783701:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05e00 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.783709:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05e00 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.783840:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493769088:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52586us (52930us total) trans 4294973457 rc 0/0 00000100:00100000:1.0:1789760755.783851:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6168 00000100:00100000:1.0:1789760755.783860:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766035476047 time 766072276469 state 0x2 ref 0 00000100:00100000:2.0:1789760755.793279:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760755.793381:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493769216 00000100:00100000:1.0:1789760755.793390:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493769216 00000100:00100000:1.0:1789760755.793453:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766035476047 time 766072276469 state 0x0 ref 1 00000100:00100000:1.0:1789760755.793468:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766072536885 time 766072276469 state 0x1 ref 1 00000100:00100000:1.0:1789760755.793498:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766072536885 time 766134533565 state 0x0 ref 1 00000100:00100000:1.0:1789760755.793511:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493769216:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760755.793559:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25145344, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc007 00010000:00010000:1.0:1789760755.793568:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05e07 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.793584:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760755.793588:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760755.793595:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05e07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.793608:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05e07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760755.894897:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973458 is committed 00002000:00010000:3.0:1789760755.895014:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25145344 -> 25149440 00010000:00010000:3.0:1789760755.895030:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05e07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.895042:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05e07 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.895050:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05e07 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.895056:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760755.895060:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05e07 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.895068:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05e07 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.895086:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05e07 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.895093:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e05e07 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.895176:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493769216:12345-192.168.203.60@tcp:4:dd.0 Request processed in 101664us (101903us total) trans 4294973458 rc 0/0 00000100:00100000:3.0:1789760755.895186:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6169 00000100:00100000:3.0:1789760755.895196:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766072536885 time 766134533565 state 0x2 ref 0 00000100:00100000:3.0:1789760755.908063:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760755.908168:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493769344 00000100:00100000:2.0:1789760755.908178:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493769344 00000100:00100000:2.0:1789760755.908232:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766072536885 time 766134533565 state 0x0 ref 1 00000100:00100000:2.0:1789760755.908248:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766134793981 time 766134533565 state 0x1 ref 1 00000100:00100000:2.0:1789760755.908281:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766134793981 time 766249316736 state 0x0 ref 1 00000100:00100000:2.0:1789760755.908293:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493769344:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760755.908337:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25149440, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc007 00010000:00010000:2.0:1789760755.908346:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05e0e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.908360:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760755.908365:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760755.908380:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05e0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.908394:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05e0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760755.949066:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25149440 -> 25153536 00010000:00010000:1.0:1789760755.949085:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05e0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.949098:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05e0e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.949107:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05e0e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.949114:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760755.949119:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05e0e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.949128:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05e0e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.949368:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05e0e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.949385:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e05e0e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.949713:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493769344:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41416us (41655us total) trans 4294973459 rc 0/0 00000100:00100000:1.0:1789760755.949724:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6170 00000100:00100000:1.0:1789760755.949734:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766134793981 time 766249316736 state 0x2 ref 0 00000001:00080000:1.0:1789760755.949776:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973459 is committed 00000100:00100000:2.0:1789760755.957418:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760755.957697:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493769472 00000100:00100000:3.0:1789760755.957706:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493769472 00000100:00100000:3.0:1789760755.957760:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766134793981 time 766249316736 state 0x0 ref 1 00000100:00100000:3.0:1789760755.957776:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766249577152 time 766249316736 state 0x1 ref 1 00000100:00100000:3.0:1789760755.957809:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766249577152 time 766298844581 state 0x0 ref 1 00000100:00100000:3.0:1789760755.957822:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493769472:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760755.958255:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25153536, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc007 00010000:00010000:3.0:1789760755.958264:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05e15 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.958280:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760755.958284:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.958290:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05e15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.958304:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05e15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760756.002291:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973460 is committed 00002000:00010000:1.0:1789760756.002605:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25153536 -> 25157632 00010000:00010000:1.0:1789760756.002620:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05e15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.002632:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05e15 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.002641:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05e15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.002648:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760756.002653:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05e15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.002670:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05e15 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.002689:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05e15 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.002696:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05e15 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.002790:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493769472:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44962us (45373us total) trans 4294973460 rc 0/0 00000100:00100000:1.0:1789760756.002800:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6171 00000100:00100000:1.0:1789760756.002811:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766249577152 time 766298844581 state 0x2 ref 0 00000100:00100000:2.0:1789760756.006240:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760756.006344:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493769600 00000100:00100000:1.0:1789760756.006354:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493769600 00000100:00100000:1.0:1789760756.006424:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766249577152 time 766298844581 state 0x0 ref 1 00000100:00100000:1.0:1789760756.006450:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766299104997 time 766298844581 state 0x1 ref 1 00000100:00100000:1.0:1789760756.006574:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766299104997 time 766347607892 state 0x0 ref 1 00000100:00100000:1.0:1789760756.006588:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493769600:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760756.006636:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25157632, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc007 00010000:00010000:1.0:1789760756.006644:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05e1c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.006660:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760756.006664:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.006670:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05e1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.006684:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05e1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760756.031410:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973461 is committed 00002000:00010000:2.0:1789760756.031458:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25157632 -> 25161728 00010000:00010000:2.0:1789760756.031482:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05e1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.031496:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05e1c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.031505:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05e1c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.031523:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760756.031528:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05e1c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.031537:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05e1c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.031556:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05e1c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.031568:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05e1c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.031662:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493769600:12345-192.168.203.60@tcp:4:dd.0 Request processed in 25071us (25426us total) trans 4294973461 rc 0/0 00000100:00100000:2.0:1789760756.031673:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6172 00000100:00100000:2.0:1789760756.031683:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766299104997 time 766347607892 state 0x2 ref 0 00000100:00100000:3.0:1789760756.039609:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760756.040784:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493769728 00000100:00100000:1.0:1789760756.040795:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493769728 00000100:00100000:1.0:1789760756.040852:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766299104997 time 766347607892 state 0x0 ref 1 00000100:00100000:1.0:1789760756.040868:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766347868308 time 766347607892 state 0x1 ref 1 00000100:00100000:1.0:1789760756.040902:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766347868308 time 766381937361 state 0x0 ref 1 00000100:00100000:1.0:1789760756.040915:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493769728:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760756.041017:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25161728, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc007 00010000:00010000:1.0:1789760756.041026:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05e23 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.041042:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760756.041046:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.041051:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05e23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.041069:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05e23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760756.085022:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973462 is committed 00002000:00010000:2.0:1789760756.085156:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25161728 -> 25165824 00010000:00010000:2.0:1789760756.085182:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05e23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.085196:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05e23 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.085205:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05e23 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.085213:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760756.085218:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05e23 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.085227:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05e23 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.085245:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05e23 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.085258:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05e23 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.085467:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493769728:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44549us (45862us total) trans 4294973462 rc 0/0 00000100:00100000:2.0:1789760756.085478:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6173 00000100:00100000:2.0:1789760756.085487:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766347868308 time 766381937361 state 0x2 ref 0 00000100:00100000:0.0:1789760756.089770:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760756.090012:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493769856 00000100:00100000:2.0:1789760756.090022:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493769856 00000100:00100000:2.0:1789760756.090080:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766347868308 time 766381937361 state 0x0 ref 1 00000100:00100000:2.0:1789760756.090095:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766382197777 time 766381937361 state 0x1 ref 1 00000100:00100000:2.0:1789760756.090121:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766382197777 time 766431156756 state 0x0 ref 1 00000100:00100000:2.0:1789760756.090132:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493769856:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760756.090190:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25165824, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc007 00010000:00010000:2.0:1789760756.090200:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05e2a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.090215:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760756.090219:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.090227:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05e2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.090246:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05e2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760756.135682:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973463 is committed 00002000:00010000:3.0:1789760756.135841:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25165824 -> 25169920 00002000:00010000:3.0:1789760756.135918:0:6694:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb blocks from disk: 49159 -> 51207 00010000:00010000:3.0:1789760756.135933:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05e2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.135945:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05e2a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.135954:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05e2a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.136023:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760756.136029:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05e2a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.136038:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05e2a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.136057:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05e2a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.136065:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e05e2a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.136158:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493769856:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46023us (46394us total) trans 4294973463 rc 0/0 00000100:00100000:3.0:1789760756.136169:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6174 00000100:00100000:3.0:1789760756.136179:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766382197777 time 766431156756 state 0x2 ref 0 00000100:00100000:3.0:1789760756.140865:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760756.141011:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493769984 00000100:00100000:1.0:1789760756.141022:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493769984 00000100:00100000:1.0:1789760756.141076:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766382197777 time 766431156756 state 0x0 ref 1 00000100:00100000:1.0:1789760756.141095:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766431417172 time 766431156756 state 0x1 ref 1 00000100:00100000:1.0:1789760756.141128:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766431417172 time 766482163772 state 0x0 ref 1 00000100:00100000:1.0:1789760756.141141:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493769984:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760756.141200:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25169920, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:1.0:1789760756.141210:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e31 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.141224:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760756.141229:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.141235:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.141249:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760756.192026:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973464 is committed 00002000:00010000:3.0:1789760756.192289:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25169920 -> 25174016 00010000:00010000:3.0:1789760756.192305:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.192318:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e31 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.192328:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e31 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.192336:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760756.192341:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e31 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.192352:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e31 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.192385:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e31 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.192404:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e31 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.192579:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493769984:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51435us (51729us total) trans 4294973464 rc 0/0 00000100:00100000:3.0:1789760756.192591:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6175 00000100:00100000:3.0:1789760756.192602:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766431417172 time 766482163772 state 0x2 ref 0 00000100:00100000:1.0:1789760756.196642:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760756.196729:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493770112 00000100:00100000:2.0:1789760756.196740:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493770112 00000100:00100000:2.0:1789760756.197295:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766431417172 time 766482163772 state 0x0 ref 1 00000100:00100000:2.0:1789760756.197314:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766482424188 time 766482163772 state 0x1 ref 1 00000100:00100000:2.0:1789760756.197349:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766482424188 time 766538384771 state 0x0 ref 1 00000100:00100000:2.0:1789760756.197378:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493770112:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760756.197856:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25174016, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:2.0:1789760756.197867:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05e38 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.197885:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760756.197890:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.197896:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05e38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.197910:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05e38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760756.257529:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25174016 -> 25178112 00010000:00010000:0.0:1789760756.257549:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05e38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.257562:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05e38 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.257578:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05e38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.257586:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760756.257591:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05e38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.257601:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05e38 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.257621:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05e38 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.257629:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e05e38 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760756.257722:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493770112:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60356us (61085us total) trans 4294973465 rc 0/0 00000100:00100000:0.0:1789760756.257733:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6176 00000100:00100000:0.0:1789760756.257741:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766482424188 time 766538384771 state 0x2 ref 0 00000001:00080000:2.0:1789760756.258345:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973465 is committed 00000100:00100000:0.0:1789760756.262571:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760756.262679:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493770240 00000100:00100000:3.0:1789760756.262688:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493770240 00000100:00100000:3.0:1789760756.262743:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766482424188 time 766538384771 state 0x0 ref 1 00000100:00100000:3.0:1789760756.262757:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766538645187 time 766538384771 state 0x1 ref 1 00000100:00100000:3.0:1789760756.262848:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766538645187 time 766603883130 state 0x0 ref 1 00000100:00100000:3.0:1789760756.262861:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493770240:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760756.262917:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25178112, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:3.0:1789760756.262926:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05e3f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.262942:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760756.262947:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.262954:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05e3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.263009:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05e3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.282771:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760756.295456:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973466 is committed 00002000:00010000:2.0:1789760756.295662:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25178112 -> 25182208 00010000:00010000:2.0:1789760756.295677:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05e3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.295689:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05e3f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.295697:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05e3f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.295704:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760756.295709:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05e3f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.295716:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05e3f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.295735:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05e3f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.295743:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e05e3f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.295995:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493770240:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33095us (33392us total) trans 4294973466 rc 0/0 00000100:00100000:2.0:1789760756.296008:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6177 00000100:00100000:2.0:1789760756.296017:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766538645187 time 766603883130 state 0x2 ref 0 00000100:00100000:3.0:1789760756.299913:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760756.300106:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493770368 00000100:00100000:2.0:1789760756.300116:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493770368 00000100:00100000:2.0:1789760756.300174:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766538645187 time 766603883130 state 0x0 ref 1 00000100:00100000:2.0:1789760756.300189:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766604143546 time 766603883130 state 0x1 ref 1 00000100:00100000:2.0:1789760756.300211:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766604143546 time 766641247146 state 0x0 ref 1 00000100:00100000:2.0:1789760756.300223:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493770368:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760756.300271:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25182208, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:2.0:1789760756.300280:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05e46 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.300296:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760756.300301:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.300307:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05e46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.300321:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05e46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760756.351847:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25182208 -> 25186304 00010000:00010000:2.0:1789760756.351866:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05e46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.351879:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05e46 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.351889:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05e46 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.351897:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760756.351902:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05e46 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.351911:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05e46 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.351930:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05e46 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.351938:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05e46 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.352041:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493770368:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51815us (52133us total) trans 4294973467 rc 0/0 00000100:00100000:2.0:1789760756.352051:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6178 00000100:00100000:2.0:1789760756.352061:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766604143546 time 766641247146 state 0x2 ref 0 00000001:00080000:0.0:1789760756.353475:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973467 is committed 00000100:00100000:0.0:1789760756.360221:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760756.360338:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493770496 00000100:00100000:1.0:1789760756.360348:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493770496 00000100:00100000:1.0:1789760756.360490:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766604143546 time 766641247146 state 0x0 ref 1 00000100:00100000:1.0:1789760756.360505:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766641507562 time 766641247146 state 0x1 ref 1 00000100:00100000:1.0:1789760756.360532:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766641507562 time 766701567976 state 0x0 ref 1 00000100:00100000:1.0:1789760756.360544:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493770496:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760756.360586:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25186304, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:1.0:1789760756.360595:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e4d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.360610:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760756.360614:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.360620:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.360633:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760756.391794:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25186304 -> 25190400 00010000:00010000:1.0:1789760756.391813:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.391824:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e4d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.391833:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e4d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.391840:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760756.391845:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e4d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.391854:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e4d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.391873:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e4d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.391881:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e4d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.392001:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493770496:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31453us (31783us total) trans 4294973468 rc 0/0 00000100:00100000:1.0:1789760756.392011:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6179 00000100:00100000:1.0:1789760756.392021:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766641507562 time 766701567976 state 0x2 ref 0 00000001:00080000:3.0:1789760756.392096:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973468 is committed 00000100:00100000:0.0:1789760756.397380:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760756.398051:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493770624 00000100:00100000:1.0:1789760756.398062:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493770624 00000100:00100000:1.0:1789760756.398256:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766641507562 time 766701567976 state 0x0 ref 1 00000100:00100000:1.0:1789760756.398272:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766701828392 time 766701567976 state 0x1 ref 1 00000100:00100000:1.0:1789760756.398386:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766701828392 time 766739420560 state 0x0 ref 1 00000100:00100000:1.0:1789760756.398402:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493770624:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760756.398450:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25190400, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:1.0:1789760756.398572:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e54 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.398587:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760756.398591:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.398597:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.398611:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760756.439850:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25190400 -> 25194496 00010000:00010000:1.0:1789760756.439870:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760756.439877:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973469 is committed 00010000:00010000:1.0:1789760756.439881:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e54 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.439889:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e54 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.439896:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760756.439901:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e54 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.439910:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e54 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.439930:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e54 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.439939:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e54 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.440065:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493770624:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41662us (42698us total) trans 4294973469 rc 0/0 00000100:00100000:1.0:1789760756.440076:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6180 00000100:00100000:1.0:1789760756.440085:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766701828392 time 766739420560 state 0x2 ref 0 00000100:00100000:2.0:1789760756.447694:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760756.448025:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493770752 00000100:00100000:3.0:1789760756.448036:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493770752 00000100:00100000:3.0:1789760756.448091:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766701828392 time 766739420560 state 0x0 ref 1 00000100:00100000:3.0:1789760756.448105:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766739680976 time 766739420560 state 0x1 ref 1 00000100:00100000:3.0:1789760756.448138:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766739680976 time 766789173090 state 0x0 ref 1 00000100:00100000:3.0:1789760756.448151:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493770752:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760756.448197:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25194496, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:3.0:1789760756.448206:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05e5b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.448221:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760756.448225:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.448232:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05e5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.448245:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05e5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760756.481684:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973470 is committed 00002000:00010000:3.0:1789760756.481864:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25194496 -> 25198592 00010000:00010000:3.0:1789760756.481880:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05e5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.481893:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05e5b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.481901:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05e5b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.481908:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760756.481913:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05e5b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.481921:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05e5b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.482084:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05e5b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.482092:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05e5b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.482286:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493770752:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34131us (34594us total) trans 4294973470 rc 0/0 00000100:00100000:2.0:1789760756.482297:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6181 00000100:00100000:2.0:1789760756.482307:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766739680976 time 766789173090 state 0x2 ref 0 00000100:00100000:0.0:1789760756.492202:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760756.492302:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493770880 00000100:00100000:2.0:1789760756.492311:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493770880 00000100:00100000:2.0:1789760756.492376:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766739680976 time 766789173090 state 0x0 ref 1 00000100:00100000:2.0:1789760756.492391:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766789433506 time 766789173090 state 0x1 ref 1 00000100:00100000:2.0:1789760756.492422:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766789433506 time 766833457093 state 0x0 ref 1 00000100:00100000:2.0:1789760756.492435:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb05050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493770880:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760756.492482:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25198592, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:2.0:1789760756.492490:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05e62 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.492505:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760756.492509:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.492516:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05e62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.492530:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05e62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760756.526202:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973471 is committed 00002000:00010000:2.0:1789760756.526396:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25198592 -> 25202688 00010000:00010000:2.0:1789760756.526411:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05e62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.526441:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05e62 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.526450:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05e62 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.526458:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760756.526462:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05e62 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.526471:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05e62 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.526491:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05e62 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.526499:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e05e62 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.527346:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb05050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493770880:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34907us (35148us total) trans 4294973471 rc 0/0 00000100:00100000:2.0:1789760756.527359:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6182 00000100:00100000:2.0:1789760756.527380:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766789433506 time 766833457093 state 0x2 ref 0 00000100:00100000:0.0:1789760756.532006:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760756.532132:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493771008 00000100:00100000:1.0:1789760756.532141:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493771008 00000100:00100000:1.0:1789760756.532194:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766789433506 time 766833457093 state 0x0 ref 1 00000100:00100000:1.0:1789760756.532209:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766833717509 time 766833457093 state 0x1 ref 1 00000100:00100000:1.0:1789760756.532239:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766833717509 time 766873274617 state 0x0 ref 1 00000100:00100000:1.0:1789760756.532252:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493771008:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760756.532302:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25202688, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:1.0:1789760756.532311:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e69 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.532326:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760756.532330:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.532336:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.532351:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760756.632021:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25202688 -> 25206784 00010000:00010000:1.0:1789760756.632041:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.632055:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e69 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.632064:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.632072:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760756.632077:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.632086:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e69 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.632107:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e69 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760756.632110:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973472 is committed 00010000:00010000:1.0:1789760756.632119:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e69 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.632214:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493771008:12345-192.168.203.60@tcp:4:dd.0 Request processed in 99960us (100212us total) trans 4294973472 rc 0/0 00000100:00100000:1.0:1789760756.632224:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6183 00000100:00100000:1.0:1789760756.632236:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766833717509 time 766873274617 state 0x2 ref 0 00000100:00100000:3.0:1789760756.637473:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760756.637554:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493771136 00000100:00100000:1.0:1789760756.637563:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493771136 00000100:00100000:1.0:1789760756.637623:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766833717509 time 766873274617 state 0x0 ref 1 00000100:00100000:1.0:1789760756.637639:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766873535033 time 766873274617 state 0x1 ref 1 00000100:00100000:1.0:1789760756.637673:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766873535033 time 766978707629 state 0x0 ref 1 00000100:00100000:1.0:1789760756.637685:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493771136:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760756.637732:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25206784, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xc807 00010000:00010000:1.0:1789760756.637750:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e70 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.637765:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760756.637773:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.637780:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.637794:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760756.683584:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973473 is committed 00002000:00010000:1.0:1789760756.683689:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25206784 -> 25210880 00002000:00010000:1.0:1789760756.683694:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760755 -> 1789760756 00002000:00010000:1.0:1789760756.683697:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760755 -> 1789760756 00010000:00010000:1.0:1789760756.683710:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.683722:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e70 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.683732:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e70 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.683740:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760756.689638:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e70 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.689656:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e70 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.689681:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e70 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.689691:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e70 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.690609:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493771136:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52917us (53138us total) trans 4294973473 rc 0/0 00000100:00100000:2.0:1789760756.690621:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6184 00000100:00100000:2.0:1789760756.690632:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766873535033 time 766978707629 state 0x2 ref 0 00000100:00100000:2.0:1789760756.698688:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760756.698851:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493771264 00000100:00100000:0.0:1789760756.698861:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493771264 00000100:00100000:0.0:1789760756.698916:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766873535033 time 766978707629 state 0x0 ref 1 00000100:00100000:0.0:1789760756.698942:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766978968045 time 766978707629 state 0x1 ref 1 00000100:00100000:0.0:1789760756.699049:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766978968045 time 767040083982 state 0x0 ref 1 00000100:00100000:0.0:1789760756.699064:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493771264:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760756.699110:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25210880, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:0.0:1789760756.699119:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05e77 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.699135:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:0.0:1789760756.699139:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.699145:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05e77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.699158:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05e77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760756.766107:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973474 is committed 00002000:00010000:0.0:1789760756.766174:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25210880 -> 25214976 00010000:00010000:0.0:1789760756.766189:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05e77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.766203:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05e77 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.766212:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05e77 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.766220:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760756.766225:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05e77 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.766235:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05e77 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.766254:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05e77 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.766262:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05e77 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760756.766359:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493771264:12345-192.168.203.60@tcp:4:dd.0 Request processed in 67293us (67674us total) trans 4294973474 rc 0/0 00000100:00100000:0.0:1789760756.766389:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6185 00000100:00100000:0.0:1789760756.766398:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766978968045 time 767040083982 state 0x2 ref 0 00000100:00100000:0.0:1789760756.775833:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760756.775934:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493771392 00000100:00100000:2.0:1789760756.775944:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493771392 00000100:00100000:2.0:1789760756.776060:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 766978968045 time 767040083982 state 0x0 ref 1 00000100:00100000:2.0:1789760756.776077:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767040344398 time 767040083982 state 0x1 ref 1 00000100:00100000:2.0:1789760756.776099:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767040344398 time 767117135341 state 0x0 ref 1 00000100:00100000:2.0:1789760756.776111:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493771392:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760756.776158:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25214976, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:2.0:1789760756.776167:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05e7e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.776182:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:2.0:1789760756.776187:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.776194:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05e7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.776208:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05e7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760756.808865:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25214976 -> 25219072 00010000:00010000:2.0:1789760756.808883:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05e7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.808897:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05e7e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.808906:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05e7e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.808914:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760756.808920:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05e7e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.808929:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05e7e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.808947:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05e7e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.808955:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05e7e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.809055:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493771392:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32941us (33226us total) trans 4294973475 rc 0/0 00000100:00100000:2.0:1789760756.809065:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6186 00000100:00100000:2.0:1789760756.809074:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767040344398 time 767117135341 state 0x2 ref 0 00000001:00080000:0.0:1789760756.810821:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973475 is committed 00000100:00100000:2.0:1789760756.816810:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760756.818393:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493771520 00000100:00100000:1.0:1789760756.818405:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493771520 00000100:00100000:1.0:1789760756.818476:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767040344398 time 767117135341 state 0x0 ref 1 00000100:00100000:1.0:1789760756.818492:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767117395757 time 767117135341 state 0x1 ref 1 00000100:00100000:1.0:1789760756.818528:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767117395757 time 767159563216 state 0x0 ref 1 00000100:00100000:1.0:1789760756.818541:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493771520:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760756.818590:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25219072, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:1.0:1789760756.818599:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e85 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.818614:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760756.818619:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.818624:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.818639:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760756.889921:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25219072 -> 25223168 00010000:00010000:1.0:1789760756.890071:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.890087:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e85 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.890097:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e85 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.890105:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760756.890109:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e85 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.890119:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e85 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.890138:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e85 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.890146:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05e85 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.890297:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493771520:12345-192.168.203.60@tcp:4:dd.0 Request processed in 71753us (73491us total) trans 4294973476 rc 0/0 00000100:00100000:1.0:1789760756.890308:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6187 00000100:00100000:1.0:1789760756.890317:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767117395757 time 767159563216 state 0x2 ref 0 00000001:00080000:0.0:1789760756.890345:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973476 is committed 00000100:00100000:0.0:1789760756.897406:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760756.898342:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493771648 00000100:00100000:1.0:1789760756.898354:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493771648 00000100:00100000:1.0:1789760756.898429:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767117395757 time 767159563216 state 0x0 ref 1 00000100:00100000:1.0:1789760756.898445:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767159823632 time 767159563216 state 0x1 ref 1 00000100:00100000:1.0:1789760756.898487:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767159823632 time 767239522340 state 0x0 ref 1 00000100:00100000:1.0:1789760756.898501:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493771648:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760756.898551:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25223168, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:1.0:1789760756.898562:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e8c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.898577:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760756.898582:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.898588:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.898602:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760756.980809:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973477 is committed 00002000:00010000:1.0:1789760756.980836:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25223168 -> 25227264 00010000:00010000:1.0:1789760756.980936:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.980948:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e8c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.980958:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e8c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.981039:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760756.981044:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e8c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.981052:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e8c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.981144:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e8c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.981154:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05e8c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760756.981324:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493771648:12345-192.168.203.60@tcp:4:dd.0 Request processed in 82820us (83921us total) trans 4294973477 rc 0/0 00000100:00100000:0.0:1789760756.981336:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6188 00000100:00100000:0.0:1789760756.981347:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767159823632 time 767239522340 state 0x2 ref 0 00000100:00100000:3.0:1789760756.985873:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760756.985998:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493771776 00000100:00100000:0.0:1789760756.986009:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493771776 00000100:00100000:0.0:1789760756.986066:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767159823632 time 767239522340 state 0x0 ref 1 00000100:00100000:0.0:1789760756.986082:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767239782756 time 767239522340 state 0x1 ref 1 00000100:00100000:0.0:1789760756.986110:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767239782756 time 767327145851 state 0x0 ref 1 00000100:00100000:0.0:1789760756.986124:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493771776:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760756.986172:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25227264, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:0.0:1789760756.986181:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05e93 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.986197:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:0.0:1789760756.986202:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.986207:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05e93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.986225:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05e93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760757.034590:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973478 is committed 00002000:00010000:0.0:1789760757.034797:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25227264 -> 25231360 00010000:00010000:0.0:1789760757.034812:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05e93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.034824:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05e93 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.034833:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05e93 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.034840:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760757.034855:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05e93 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.034864:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05e93 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.034883:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05e93 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.034890:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05e93 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.035028:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493771776:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48902us (49159us total) trans 4294973478 rc 0/0 00000100:00100000:0.0:1789760757.035039:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6189 00000100:00100000:0.0:1789760757.035049:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767239782756 time 767327145851 state 0x2 ref 0 00000100:00100000:2.0:1789760757.038928:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760757.039029:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493771904 00000100:00100000:3.0:1789760757.039039:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493771904 00000100:00100000:3.0:1789760757.039111:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767239782756 time 767327145851 state 0x0 ref 1 00000100:00100000:3.0:1789760757.039125:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767327406267 time 767327145851 state 0x1 ref 1 00000100:00100000:3.0:1789760757.039148:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767327406267 time 767380183022 state 0x0 ref 1 00000100:00100000:3.0:1789760757.039159:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493771904:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760757.039206:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25231360, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:3.0:1789760757.039214:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05e9a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.039229:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760757.039234:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.039240:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05e9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.039253:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05e9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760757.039630:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760757.075951:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973479 is committed 00002000:00010000:2.0:1789760757.076119:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25231360 -> 25235456 00010000:00010000:2.0:1789760757.076134:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05e9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.076146:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05e9a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.076154:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05e9a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.076161:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760757.076166:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05e9a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.076175:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05e9a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.076194:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05e9a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.076201:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e05e9a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.076399:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493771904:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37236us (37476us total) trans 4294973479 rc 0/0 00000100:00100000:2.0:1789760757.076409:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6190 00000100:00100000:2.0:1789760757.077144:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767327406267 time 767380183022 state 0x2 ref 0 00000100:00100000:3.0:1789760757.080950:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760757.081167:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493772032 00000100:00100000:1.0:1789760757.081177:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493772032 00000100:00100000:1.0:1789760757.081231:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767327406267 time 767380183022 state 0x0 ref 1 00000100:00100000:1.0:1789760757.081247:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767380443438 time 767380183022 state 0x1 ref 1 00000100:00100000:1.0:1789760757.081314:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767380443438 time 767422349484 state 0x0 ref 1 00000100:00100000:1.0:1789760757.081338:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493772032:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760757.081402:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25235456, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:1.0:1789760757.081411:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ea1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.081427:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760757.081432:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.081439:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ea1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.081454:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ea1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760757.162754:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973480 is committed 00002000:00010000:1.0:1789760757.162851:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25235456 -> 25239552 00010000:00010000:1.0:1789760757.162866:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ea1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.162889:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ea1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.162900:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ea1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.162908:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760757.162912:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ea1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.162921:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ea1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.162940:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ea1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.162948:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ea1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.163059:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493772032:12345-192.168.203.60@tcp:4:dd.0 Request processed in 81728us (82114us total) trans 4294973480 rc 0/0 00000100:00100000:1.0:1789760757.163069:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6191 00000100:00100000:1.0:1789760757.163079:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767380443438 time 767422349484 state 0x2 ref 0 00000100:00100000:3.0:1789760757.166529:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760757.166663:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493772160 00000100:00100000:1.0:1789760757.166674:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493772160 00000100:00100000:1.0:1789760757.166731:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767380443438 time 767422349484 state 0x0 ref 1 00000100:00100000:1.0:1789760757.166745:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767422609900 time 767422349484 state 0x1 ref 1 00000100:00100000:1.0:1789760757.166778:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767422609900 time 767507813915 state 0x0 ref 1 00000100:00100000:1.0:1789760757.166790:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493772160:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760757.166839:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25239552, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:1.0:1789760757.166848:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05ea8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.166864:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760757.166869:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.166884:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05ea8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.166899:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05ea8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760757.218180:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973481 is committed 00002000:00010000:2.0:1789760757.218251:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25239552 -> 25243648 00010000:00010000:2.0:1789760757.218267:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05ea8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.218280:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05ea8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.218289:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05ea8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.218296:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760757.218301:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05ea8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.218311:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05ea8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.218329:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05ea8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.218341:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e05ea8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.218667:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493772160:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51873us (52142us total) trans 4294973481 rc 0/0 00000100:00100000:2.0:1789760757.218678:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6192 00000100:00100000:2.0:1789760757.218688:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767422609900 time 767507813915 state 0x2 ref 0 00000100:00100000:2.0:1789760757.222321:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760757.222408:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493772288 00000100:00100000:0.0:1789760757.222425:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493772288 00000100:00100000:0.0:1789760757.222478:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767422609900 time 767507813915 state 0x0 ref 1 00000100:00100000:0.0:1789760757.222493:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767508074331 time 767507813915 state 0x1 ref 1 00000100:00100000:0.0:1789760757.222532:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767508074331 time 767563567735 state 0x0 ref 1 00000100:00100000:0.0:1789760757.222545:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493772288:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760757.222590:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25243648, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:0.0:1789760757.222599:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05eaf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.222615:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:0.0:1789760757.222619:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.222625:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05eaf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.222640:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05eaf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760757.253727:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973482 is committed 00002000:00010000:0.0:1789760757.253896:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25243648 -> 25247744 00010000:00010000:0.0:1789760757.253911:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05eaf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.253925:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05eaf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.254030:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05eaf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.254038:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760757.254042:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05eaf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.254051:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05eaf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.254070:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05eaf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.254078:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e05eaf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.254315:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493772288:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31766us (31997us total) trans 4294973482 rc 0/0 00000100:00100000:0.0:1789760757.254327:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6193 00000100:00100000:0.0:1789760757.254336:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767508074331 time 767563567735 state 0x2 ref 0 00000100:00100000:3.0:1789760757.264715:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760757.267321:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493772416 00000100:00100000:2.0:1789760757.267336:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493772416 00000100:00100000:2.0:1789760757.267405:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767508074331 time 767563567735 state 0x0 ref 1 00000100:00100000:2.0:1789760757.267923:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767563828151 time 767563567735 state 0x1 ref 1 00000100:00100000:2.0:1789760757.268037:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767563828151 time 767609072031 state 0x0 ref 1 00000100:00100000:2.0:1789760757.268053:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493772416:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760757.268389:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25247744, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:2.0:1789760757.268399:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05eb6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.268424:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:2.0:1789760757.268429:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760757.268436:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05eb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.268451:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05eb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.300684:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973483 is committed 00002000:00010000:2.0:1789760757.300913:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25247744 -> 25251840 00010000:00010000:2.0:1789760757.300929:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05eb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.300941:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05eb6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.300951:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05eb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.300979:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760757.300984:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05eb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.300993:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05eb6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.301013:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05eb6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.301020:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05eb6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.301211:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493772416:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33157us (36507us total) trans 4294973483 rc 0/0 00000100:00100000:2.0:1789760757.301222:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6194 00000100:00100000:2.0:1789760757.301233:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767563828151 time 767609072031 state 0x2 ref 0 00000100:00100000:0.0:1789760757.304582:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760757.304697:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493772544 00000100:00100000:1.0:1789760757.304707:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493772544 00000100:00100000:1.0:1789760757.304762:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767563828151 time 767609072031 state 0x0 ref 1 00000100:00100000:1.0:1789760757.304777:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767609332447 time 767609072031 state 0x1 ref 1 00000100:00100000:1.0:1789760757.304841:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767609332447 time 767645876332 state 0x0 ref 1 00000100:00100000:1.0:1789760757.304855:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493772544:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760757.304910:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25251840, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:1.0:1789760757.304919:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ebd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.304934:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760757.304938:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.304944:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ebd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.304957:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ebd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.309433:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760757.346998:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973484 is committed 00002000:00010000:1.0:1789760757.347123:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25251840 -> 25255936 00010000:00010000:1.0:1789760757.347139:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ebd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.347153:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ebd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.347162:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ebd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.347171:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760757.347176:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ebd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.347185:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ebd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.347203:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ebd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.347211:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ebd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.347306:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493772544:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42450us (42729us total) trans 4294973484 rc 0/0 00000100:00100000:1.0:1789760757.347332:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6195 00000100:00100000:1.0:1789760757.347341:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767609332447 time 767645876332 state 0x2 ref 0 00000100:00100000:3.0:1789760757.354458:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760757.354702:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493772672 00000100:00100000:1.0:1789760757.354716:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493772672 00000100:00100000:1.0:1789760757.354797:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767609332447 time 767645876332 state 0x0 ref 1 00000100:00100000:1.0:1789760757.354817:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767646136748 time 767645876332 state 0x1 ref 1 00000100:00100000:1.0:1789760757.355069:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767646136748 time 767696102270 state 0x0 ref 1 00000100:00100000:1.0:1789760757.355086:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493772672:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760757.355153:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25255936, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:1.0:1789760757.355164:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ec4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.355184:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760757.355190:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.355198:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ec4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.355216:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ec4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760757.391503:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973485 is committed 00002000:00010000:2.0:1789760757.391640:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25255936 -> 25260032 00010000:00010000:2.0:1789760757.391660:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ec4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.391676:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ec4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.391687:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ec4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.391695:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760757.391700:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ec4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.391709:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ec4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.391733:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ec4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.391747:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05ec4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.391884:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493772672:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36795us (37435us total) trans 4294973485 rc 0/0 00000100:00100000:2.0:1789760757.391896:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6196 00000100:00100000:2.0:1789760757.391908:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767646136748 time 767696102270 state 0x2 ref 0 00000100:00100000:3.0:1789760757.395414:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760757.395510:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493772800 00000100:00100000:0.0:1789760757.395522:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493772800 00000100:00100000:0.0:1789760757.395588:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767646136748 time 767696102270 state 0x0 ref 1 00000100:00100000:0.0:1789760757.395606:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767696362686 time 767696102270 state 0x1 ref 1 00000100:00100000:0.0:1789760757.395638:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767696362686 time 767736673955 state 0x0 ref 1 00000100:00100000:0.0:1789760757.395650:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493772800:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760757.395712:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25260032, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:0.0:1789760757.395723:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05ecb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.395742:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:0.0:1789760757.395747:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.395755:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05ecb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.395779:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05ecb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760757.428814:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973486 is committed 00002000:00010000:0.0:1789760757.429025:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25260032 -> 25264128 00010000:00010000:0.0:1789760757.429044:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05ecb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.429060:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05ecb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.429071:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05ecb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.429079:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760757.429085:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05ecb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.429095:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05ecb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.429119:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05ecb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.429127:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e05ecb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.429281:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493772800:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33624us (33876us total) trans 4294973486 rc 0/0 00000100:00100000:0.0:1789760757.429293:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6197 00000100:00100000:0.0:1789760757.429305:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767696362686 time 767736673955 state 0x2 ref 0 00000100:00100000:0.0:1789760757.432375:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760757.432506:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493772928 00000100:00100000:2.0:1789760757.432519:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493772928 00000100:00100000:2.0:1789760757.432594:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767696362686 time 767736673955 state 0x0 ref 1 00000100:00100000:2.0:1789760757.432613:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767736934371 time 767736673955 state 0x1 ref 1 00000100:00100000:2.0:1789760757.432656:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767736934371 time 767773691426 state 0x0 ref 1 00000100:00100000:2.0:1789760757.432672:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493772928:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760757.432737:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25264128, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:2.0:1789760757.432748:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05ed2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.432768:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:2.0:1789760757.432773:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760757.432781:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05ed2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.432798:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05ed2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760757.458178:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973487 is committed 00002000:00010000:2.0:1789760757.458201:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25264128 -> 25268224 00010000:00010000:2.0:1789760757.458219:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05ed2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.458231:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05ed2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.458238:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05ed2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.458244:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760757.458248:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05ed2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.458256:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05ed2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.458274:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05ed2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.458279:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05ed2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.458576:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493772928:12345-192.168.203.60@tcp:4:dd.0 Request processed in 25724us (26054us total) trans 4294973487 rc 0/0 00000100:00100000:2.0:1789760757.458594:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6198 00000100:00100000:2.0:1789760757.458608:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767736934371 time 767773691426 state 0x2 ref 0 00000100:00100000:3.0:1789760757.461796:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760757.462075:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493773056 00000100:00100000:1.0:1789760757.462088:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493773056 00000100:00100000:1.0:1789760757.462166:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767736934371 time 767773691426 state 0x0 ref 1 00000100:00100000:1.0:1789760757.462185:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767773951842 time 767773691426 state 0x1 ref 1 00000100:00100000:1.0:1789760757.462222:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767773951842 time 767803257178 state 0x0 ref 1 00000100:00100000:1.0:1789760757.462236:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493773056:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760757.462300:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25268224, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:1.0:1789760757.462310:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ed9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.462328:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760757.462333:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.462340:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ed9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.462367:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ed9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760757.497647:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973488 is committed 00002000:00010000:3.0:1789760757.497783:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25268224 -> 25272320 00010000:00010000:3.0:1789760757.497804:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ed9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.497819:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ed9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.497834:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ed9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.498129:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760757.498136:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ed9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.498148:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ed9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.498175:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ed9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.498185:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ed9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.498307:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493773056:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36064us (36517us total) trans 4294973488 rc 0/0 00000100:00100000:3.0:1789760757.498323:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6199 00000100:00100000:3.0:1789760757.498336:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767773951842 time 767803257178 state 0x2 ref 0 00000100:00100000:3.0:1789760757.500808:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760757.501029:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493773184 00000100:00100000:1.0:1789760757.501041:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493773184 00000100:00100000:1.0:1789760757.501105:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767773951842 time 767803257178 state 0x0 ref 1 00000100:00100000:1.0:1789760757.501120:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767803517594 time 767803257178 state 0x1 ref 1 00000100:00100000:1.0:1789760757.501150:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767803517594 time 767842184707 state 0x0 ref 1 00000100:00100000:1.0:1789760757.501168:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493773184:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760757.501236:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25272320, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:1.0:1789760757.501245:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05ee0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.501262:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760757.501266:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.501272:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05ee0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.501283:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05ee0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760757.561224:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973489 is committed 00002000:00010000:2.0:1789760757.561332:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25272320 -> 25276416 00010000:00010000:2.0:1789760757.561351:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05ee0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.561373:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05ee0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.561383:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05ee0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.561391:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760757.561396:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05ee0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.561407:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05ee0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.561547:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05ee0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.561561:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05ee0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.561723:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493773184:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60549us (60920us total) trans 4294973489 rc 0/0 00000100:00100000:2.0:1789760757.561737:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6200 00000100:00100000:2.0:1789760757.561749:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767803517594 time 767842184707 state 0x2 ref 0 00000100:00100000:0.0:1789760757.566563:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760757.567191:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493773312 00000100:00100000:2.0:1789760757.567205:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493773312 00000100:00100000:3.0:1789760757.567516:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767803517594 time 767842184707 state 0x0 ref 1 00000100:00100000:3.0:1789760757.567540:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767842445123 time 767842184707 state 0x1 ref 1 00000100:00100000:3.0:1789760757.567630:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767842445123 time 767908664208 state 0x0 ref 1 00000100:00100000:3.0:1789760757.567646:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493773312:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760757.567712:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25276416, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:3.0:1789760757.567723:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05ee7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.567744:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760757.567749:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.567756:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05ee7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.567774:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05ee7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760757.611274:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973490 is committed 00002000:00010000:3.0:1789760757.612896:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25276416 -> 25280512 00010000:00010000:3.0:1789760757.612920:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05ee7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.612936:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05ee7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.612947:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05ee7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.612955:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760757.612990:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05ee7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.613003:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05ee7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.613029:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05ee7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.613038:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05ee7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.613223:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493773312:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45573us (46668us total) trans 4294973490 rc 0/0 00000100:00100000:3.0:1789760757.613237:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6201 00000100:00100000:3.0:1789760757.613249:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767842445123 time 767908664208 state 0x2 ref 0 00000100:00100000:3.0:1789760757.625895:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760757.628582:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493773440 00000100:00100000:2.0:1789760757.628671:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493773440 00000100:00100000:2.0:1789760757.628769:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767842445123 time 767908664208 state 0x0 ref 1 00000100:00100000:2.0:1789760757.628790:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767908924624 time 767908664208 state 0x1 ref 1 00000100:00100000:2.0:1789760757.628848:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767908924624 time 767969881691 state 0x0 ref 1 00000100:00100000:2.0:1789760757.628865:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493773440:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760757.628934:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25280512, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xc807 00010000:00010000:2.0:1789760757.628945:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05eee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.628984:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=767s v=(0 0) 00010000:00010000:2.0:1789760757.628990:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760757.628999:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05eee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.629015:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05eee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760757.686054:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25280512 -> 25284608 00002000:00010000:2.0:1789760757.686065:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760756 -> 1789760757 00002000:00010000:2.0:1789760757.686068:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760756 -> 1789760757 00010000:00010000:2.0:1789760757.686085:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05eee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.686100:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05eee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.686109:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05eee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.686118:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760757.686123:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05eee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.686133:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05eee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.686161:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05eee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.686175:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05eee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.686301:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493773440:12345-192.168.203.60@tcp:4:dd.0 Request processed in 57431us (60422us total) trans 4294973491 rc 0/0 00000100:00100000:2.0:1789760757.686316:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6202 00000100:00100000:2.0:1789760757.686329:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767908924624 time 767969881691 state 0x2 ref 0 00000001:00080000:1.0:1789760757.686499:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973491 is committed 00000100:00100000:0.0:1789760757.689953:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760757.690139:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493773568 00000100:00100000:1.0:1789760757.690153:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493773568 00000100:00100000:1.0:1789760757.690229:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767908924624 time 767969881691 state 0x0 ref 1 00000100:00100000:1.0:1789760757.690247:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767970142107 time 767969881691 state 0x1 ref 1 00000100:00100000:1.0:1789760757.690282:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767970142107 time 768031317673 state 0x0 ref 1 00000100:00100000:2.0:1789760757.690492:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493773568:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760757.690559:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25284608, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:2.0:1789760757.690571:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05ef5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.690592:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760757.690597:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760757.690604:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05ef5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.690621:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05ef5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.732937:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973492 is committed 00002000:00010000:2.0:1789760757.733131:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25284608 -> 25288704 00010000:00010000:2.0:1789760757.733150:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05ef5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.733172:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05ef5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.733182:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05ef5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.733190:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760757.733195:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05ef5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.733206:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05ef5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.733230:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05ef5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.733238:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05ef5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.733395:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493773568:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43093us (43451us total) trans 4294973492 rc 0/0 00000100:00100000:2.0:1789760757.733408:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6203 00000100:00100000:2.0:1789760757.733475:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767970142107 time 768031317673 state 0x2 ref 0 00000100:00100000:3.0:1789760757.736034:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760757.736253:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493773696 00000100:00100000:1.0:1789760757.736266:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493773696 00000100:00100000:1.0:1789760757.736335:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 767970142107 time 768031317673 state 0x0 ref 1 00000100:00100000:1.0:1789760757.736353:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768031578089 time 768031317673 state 0x1 ref 1 00000100:00100000:1.0:1789760757.736525:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768031578089 time 768077559322 state 0x0 ref 1 00000100:00100000:1.0:1789760757.736541:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493773696:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760757.736621:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25288704, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:1.0:1789760757.736632:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05efc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.736651:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:1.0:1789760757.736656:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.736663:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05efc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.736680:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05efc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760757.766827:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25288704 -> 25292800 00010000:00010000:1.0:1789760757.766853:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05efc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.766868:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05efc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.766878:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05efc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.766888:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760757.766893:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05efc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.766904:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05efc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.766930:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05efc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.766938:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e05efc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760757.767132:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973493 is committed 00000100:00100000:1.0:1789760757.767194:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493773696:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30648us (31162us total) trans 4294973493 rc 0/0 00000100:00100000:1.0:1789760757.767208:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6204 00000100:00100000:1.0:1789760757.767220:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768031578089 time 768077559322 state 0x2 ref 0 00000100:00100000:1.0:1789760757.777254:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760757.777380:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493773824 00000100:00100000:3.0:1789760757.777394:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493773824 00000100:00100000:3.0:1789760757.777479:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768031578089 time 768077559322 state 0x0 ref 1 00000100:00100000:3.0:1789760757.777499:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768077819738 time 768077559322 state 0x1 ref 1 00000100:00100000:3.0:1789760757.777591:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768077819738 time 768118624673 state 0x0 ref 1 00000100:00100000:3.0:1789760757.777608:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493773824:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760757.777675:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25292800, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:3.0:1789760757.777685:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f03 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.777706:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760757.777711:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.777719:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.777735:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760757.815802:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973494 is committed 00002000:00010000:3.0:1789760757.815908:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25292800 -> 25296896 00010000:00010000:3.0:1789760757.815928:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.815951:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f03 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.816482:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.816494:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760757.816499:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.816510:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f03 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.816584:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f03 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.816593:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f03 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.816753:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493773824:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39142us (39507us total) trans 4294973494 rc 0/0 00000100:00100000:3.0:1789760757.816765:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6205 00000100:00100000:3.0:1789760757.816777:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768077819738 time 768118624673 state 0x2 ref 0 00000100:00100000:3.0:1789760757.822179:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760757.823395:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493773952 00000100:00100000:2.0:1789760757.823411:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493773952 00000100:00100000:2.0:1789760757.823502:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768077819738 time 768118624673 state 0x0 ref 1 00000100:00100000:2.0:1789760757.823570:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768118885089 time 768118624673 state 0x1 ref 1 00000100:00100000:2.0:1789760757.823608:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768118885089 time 768164643973 state 0x0 ref 1 00000100:00100000:2.0:1789760757.823624:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493773952:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760757.823710:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25296896, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:2.0:1789760757.823920:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f0a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.823942:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760757.823947:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760757.823955:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.824032:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.863908:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760757.863956:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760757.864003:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760757.864037:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493774208 00000100:00100000:3.0:1789760757.864050:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493774208 00000100:00100000:3.0:1789760757.864120:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493774336 00000100:00100000:3.0:1789760757.864124:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493774336 00000100:00100000:3.0:1789760757.864152:0:9460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 497 00000100:00100000:3.0:1789760757.864163:0:9460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2369:x1876699493774208:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:3.0:1789760757.864256:0:9460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2369:x1876699493774208:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 95us (364us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760757.864264:0:9460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 497 00000100:00100000:3.0:1789760757.864283:0:9460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 498 00000100:00100000:3.0:1789760757.864290:0:9460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2369:x1876699493774336:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:3.0:1789760757.864315:0:9460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2369:x1876699493774336:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 25us (312us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760757.864321:0:9460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 498 00000100:00100000:3.0:1789760757.864340:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493774080 00000100:00100000:3.0:1789760757.864344:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493774080 00000100:00100000:3.0:1789760757.864383:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 405 00000100:00100000:3.0:1789760757.864390:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d4a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2369:x1876699493774080:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:3.0:1789760757.864691:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d4a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2369:x1876699493774080:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 300us (733us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760757.864699:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 405 00000100:00080000:3.0:1789760757.864737:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:1.0:1789760757.872219:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973495 is committed 00002000:00010000:2.0:1789760757.872349:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25296896 -> 25300992 00010000:00010000:2.0:1789760757.872379:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.872394:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f0a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.872406:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.872480:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760757.872487:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.872497:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f0a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.872522:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f0a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.872532:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f0a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.872687:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493773952:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49060us (50515us total) trans 4294973495 rc 0/0 00000100:00100000:2.0:1789760757.872700:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6206 00000100:00100000:2.0:1789760757.872711:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768118885089 time 768164643973 state 0x2 ref 0 00000100:00100000:3.0:1789760757.887774:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760757.888098:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493774464 00000100:00100000:2.0:1789760757.888112:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493774464 00000100:00100000:2.0:1789760757.888189:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768118885089 time 768164643973 state 0x0 ref 1 00000100:00100000:2.0:1789760757.888209:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768164904389 time 768164643973 state 0x1 ref 1 00000100:00100000:0.0:1789760757.888434:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768164904389 time 768229457180 state 0x0 ref 1 00000100:00100000:0.0:1789760757.888454:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493774464:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760757.888529:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25300992, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:0.0:1789760757.888540:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05f11 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.888560:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760757.888565:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.888573:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05f11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.888590:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05f11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760757.931153:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973496 is committed 00002000:00010000:0.0:1789760757.931247:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25300992 -> 25305088 00010000:00010000:0.0:1789760757.931267:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05f11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.931283:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05f11 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.931294:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05f11 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.931302:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760757.931307:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05f11 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.931317:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05f11 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.931342:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05f11 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.931350:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05f11 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.932164:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493774464:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43708us (44396us total) trans 4294973496 rc 0/0 00000100:00100000:0.0:1789760757.932180:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6207 00000100:00100000:0.0:1789760757.932193:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768164904389 time 768229457180 state 0x2 ref 0 00000100:00100000:0.0:1789760757.940884:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760757.941269:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493774592 00000100:00100000:1.0:1789760757.941282:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493774592 00000100:00100000:0.0:1789760757.941641:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768164904389 time 768229457180 state 0x0 ref 1 00000100:00100000:0.0:1789760757.941660:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768229717596 time 768229457180 state 0x1 ref 1 00000100:00100000:0.0:1789760757.941738:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768229717596 time 768282772564 state 0x0 ref 1 00000100:00100000:0.0:1789760757.941761:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493774592:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760757.941827:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25305088, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:0.0:1789760757.941838:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05f18 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.941859:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760757.941865:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.941873:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05f18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.941890:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05f18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760757.993102:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973497 is committed 00002000:00010000:2.0:1789760757.993207:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25305088 -> 25309184 00010000:00010000:2.0:1789760757.993228:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05f18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.993242:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05f18 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.993252:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05f18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.993259:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760757.993264:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05f18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.993272:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05f18 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.993295:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05f18 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.993302:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05f18 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.993644:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493774592:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51886us (52769us total) trans 4294973497 rc 0/0 00000100:00100000:2.0:1789760757.993656:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6208 00000100:00100000:2.0:1789760757.993669:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768229717596 time 768282772564 state 0x2 ref 0 00000100:00100000:3.0:1789760757.997224:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760757.998778:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493774720 00000100:00100000:0.0:1789760757.998797:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493774720 00000100:00100000:0.0:1789760757.999039:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768229717596 time 768282772564 state 0x0 ref 1 00000100:00100000:0.0:1789760757.999059:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768283032980 time 768282772564 state 0x1 ref 1 00000100:00100000:0.0:1789760757.999138:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768283032980 time 768340172954 state 0x0 ref 1 00000100:00100000:0.0:1789760757.999153:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493774720:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760757.999216:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25309184, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:0.0:1789760757.999227:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05f1f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.999246:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760757.999251:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.999258:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05f1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.999275:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05f1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760758.043318:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973498 is committed 00002000:00010000:1.0:1789760758.043587:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25309184 -> 25313280 00010000:00010000:1.0:1789760758.043609:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05f1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.043629:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05f1f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.043639:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05f1f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.043647:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760758.043652:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05f1f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.043662:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05f1f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.043686:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05f1f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.043695:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05f1f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.043814:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493774720:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44657us (46599us total) trans 4294973498 rc 0/0 00000100:00100000:1.0:1789760758.043826:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6209 00000100:00100000:1.0:1789760758.043839:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768283032980 time 768340172954 state 0x2 ref 0 00000100:00100000:3.0:1789760758.050474:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760758.050853:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493774848 00000100:00100000:2.0:1789760758.050868:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493774848 00000100:00100000:2.0:1789760758.050942:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768283032980 time 768340172954 state 0x0 ref 1 00000100:00100000:2.0:1789760758.051059:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768340433370 time 768340172954 state 0x1 ref 1 00000100:00100000:3.0:1789760758.051189:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768340433370 time 768392221432 state 0x0 ref 1 00000100:00100000:3.0:1789760758.051205:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493774848:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760758.051269:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25313280, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:3.0:1789760758.051279:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05f26 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.051298:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760758.051303:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.051311:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05f26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.051327:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05f26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.088213:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25313280 -> 25317376 00010000:00010000:3.0:1789760758.088241:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05f26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.088256:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05f26 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.088265:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05f26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.088280:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760758.088286:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05f26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.088296:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05f26 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.088321:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05f26 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.088329:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05f26 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.088466:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493774848:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37257us (38000us total) trans 4294973499 rc 0/0 00000100:00100000:3.0:1789760758.088479:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6210 00000100:00100000:3.0:1789760758.088492:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768340433370 time 768392221432 state 0x2 ref 0 00000001:00080000:2.0:1789760758.089178:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973499 is committed 00000100:00100000:3.0:1789760758.095204:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760758.095422:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493774976 00000100:00100000:2.0:1789760758.095434:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493774976 00000100:00100000:2.0:1789760758.095506:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768340433370 time 768392221432 state 0x0 ref 1 00000100:00100000:2.0:1789760758.095536:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768392481848 time 768392221432 state 0x1 ref 1 00000100:00100000:2.0:1789760758.095619:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768392481848 time 768436654303 state 0x0 ref 1 00000100:00100000:2.0:1789760758.095633:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493774976:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760758.095692:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25317376, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:2.0:1789760758.095702:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f2d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.095720:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760758.095725:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.095744:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.095761:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760758.142343:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973500 is committed 00002000:00010000:2.0:1789760758.142625:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25317376 -> 25321472 00010000:00010000:2.0:1789760758.142643:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.142656:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f2d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.142664:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.142672:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760758.142677:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.142687:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.142710:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f2d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.142719:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f2d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.142838:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493774976:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47200us (47641us total) trans 4294973500 rc 0/0 00000100:00100000:2.0:1789760758.142853:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6211 00000100:00100000:2.0:1789760758.142866:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768392481848 time 768436654303 state 0x2 ref 0 00000100:00100000:0.0:1789760758.146478:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760758.146569:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493775104 00000100:00100000:2.0:1789760758.146600:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493775104 00000100:00100000:2.0:1789760758.146689:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768392481848 time 768436654303 state 0x0 ref 1 00000100:00100000:2.0:1789760758.146716:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768436914719 time 768436654303 state 0x1 ref 1 00000100:00100000:2.0:1789760758.146756:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768436914719 time 768487790557 state 0x0 ref 1 00000100:00100000:2.0:1789760758.146770:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493775104:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760758.146835:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25321472, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:2.0:1789760758.146846:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05f34 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.146871:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760758.146877:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.146884:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05f34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.146908:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05f34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760758.199021:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973501 is committed 00002000:00010000:1.0:1789760758.199264:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25321472 -> 25325568 00010000:00010000:1.0:1789760758.199285:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05f34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.199301:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05f34 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.199311:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05f34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.199320:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760758.199327:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05f34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.199337:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05f34 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.199370:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05f34 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.199380:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e05f34 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.199599:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493775104:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52824us (53127us total) trans 4294973501 rc 0/0 00000100:00100000:1.0:1789760758.199612:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6212 00000100:00100000:1.0:1789760758.199625:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768436914719 time 768487790557 state 0x2 ref 0 00000100:00100000:1.0:1789760758.203887:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760758.204035:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493775232 00000100:00100000:0.0:1789760758.204049:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493775232 00000100:00100000:0.0:1789760758.204120:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768436914719 time 768487790557 state 0x0 ref 1 00000100:00100000:0.0:1789760758.204138:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768488050973 time 768487790557 state 0x1 ref 1 00000100:00100000:0.0:1789760758.204175:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768488050973 time 768545209497 state 0x0 ref 1 00000100:00100000:0.0:1789760758.204192:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493775232:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760758.204255:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25325568, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:0.0:1789760758.204267:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05f3b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.204286:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760758.204291:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.204299:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05f3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.204315:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05f3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760758.245788:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973502 is committed 00002000:00010000:2.0:1789760758.245916:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25325568 -> 25329664 00010000:00010000:2.0:1789760758.246025:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05f3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.246043:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05f3b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.246053:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05f3b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.246062:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760758.246068:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05f3b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.246078:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05f3b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.246103:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05f3b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.246112:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05f3b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.246225:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493775232:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42028us (42346us total) trans 4294973502 rc 0/0 00000100:00100000:2.0:1789760758.246237:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6213 00000100:00100000:2.0:1789760758.246249:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768488050973 time 768545209497 state 0x2 ref 0 00000100:00100000:3.0:1789760758.248781:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760758.248885:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493775360 00000100:00100000:0.0:1789760758.248898:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493775360 00000100:00100000:0.0:1789760758.249232:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768488050973 time 768545209497 state 0x0 ref 1 00000100:00100000:0.0:1789760758.249251:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768545469913 time 768545209497 state 0x1 ref 1 00000100:00100000:0.0:1789760758.249496:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768545469913 time 768590529935 state 0x0 ref 1 00000100:00100000:0.0:1789760758.249513:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493775360:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760758.249579:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25329664, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:0.0:1789760758.249590:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05f42 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.249615:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760758.249620:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.249628:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05f42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.249644:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05f42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760758.278894:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973503 is committed 00002000:00010000:2.0:1789760758.279201:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25329664 -> 25333760 00010000:00010000:2.0:1789760758.279222:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05f42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.279237:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05f42 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.279247:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05f42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.279255:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760758.279261:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05f42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.279271:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05f42 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.279296:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05f42 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.279305:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e05f42 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.279447:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493775360:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29931us (30672us total) trans 4294973503 rc 0/0 00000100:00100000:2.0:1789760758.279461:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6214 00000100:00100000:2.0:1789760758.279473:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768545469913 time 768590529935 state 0x2 ref 0 00000100:00100000:0.0:1789760758.282225:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760758.282316:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493775488 00000100:00100000:3.0:1789760758.282328:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493775488 00000100:00100000:3.0:1789760758.282416:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768545469913 time 768590529935 state 0x0 ref 1 00000100:00100000:3.0:1789760758.282434:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768590790351 time 768590529935 state 0x1 ref 1 00000100:00100000:3.0:1789760758.282461:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768590790351 time 768623497282 state 0x0 ref 1 00000100:00100000:3.0:1789760758.282475:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493775488:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760758.282533:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25333760, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:3.0:1789760758.282544:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05f49 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.282563:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760758.282568:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.282576:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05f49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.282592:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05f49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760758.314928:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973504 is committed 00002000:00010000:2.0:1789760758.316092:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25333760 -> 25337856 00010000:00010000:2.0:1789760758.316117:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05f49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.316133:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05f49 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.316143:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05f49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.316152:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760758.316157:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05f49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.316168:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05f49 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.316194:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05f49 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.316204:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e05f49 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.316337:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493775488:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33856us (34118us total) trans 4294973504 rc 0/0 00000100:00100000:2.0:1789760758.316350:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6215 00000100:00100000:2.0:1789760758.316374:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768590790351 time 768623497282 state 0x2 ref 0 00000100:00100000:3.0:1789760758.321938:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760758.322045:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493775616 00000100:00100000:2.0:1789760758.322062:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493775616 00000100:00100000:2.0:1789760758.322144:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768590790351 time 768623497282 state 0x0 ref 1 00000100:00100000:2.0:1789760758.322162:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768623757698 time 768623497282 state 0x1 ref 1 00000100:00100000:2.0:1789760758.322205:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768623757698 time 768663240250 state 0x0 ref 1 00000100:00100000:2.0:1789760758.322220:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493775616:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760758.322285:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25337856, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:2.0:1789760758.322295:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f50 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.322314:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760758.322319:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.322326:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.322342:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.330098:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760758.383829:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973505 is committed 00002000:00010000:3.0:1789760758.383891:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25337856 -> 25341952 00010000:00010000:3.0:1789760758.384165:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.384179:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f50 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.384189:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.384196:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760758.384201:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.384210:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f50 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.384233:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f50 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.384242:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e05f50 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.384333:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493775616:12345-192.168.203.60@tcp:4:dd.0 Request processed in 62112us (62400us total) trans 4294973505 rc 0/0 00000100:00100000:3.0:1789760758.384343:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6216 00000100:00100000:3.0:1789760758.384362:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768623757698 time 768663240250 state 0x2 ref 0 00000100:00100000:3.0:1789760758.391280:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760758.392532:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493775744 00000100:00100000:2.0:1789760758.392544:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493775744 00000100:00100000:2.0:1789760758.392605:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768623757698 time 768663240250 state 0x0 ref 1 00000100:00100000:2.0:1789760758.392621:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768663500666 time 768663240250 state 0x1 ref 1 00000100:00100000:2.0:1789760758.392658:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768663500666 time 768733693587 state 0x0 ref 1 00000100:00100000:2.0:1789760758.392671:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493775744:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760758.392715:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25341952, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:2.0:1789760758.392723:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05f57 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.392739:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760758.392743:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.392749:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05f57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.392762:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05f57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760758.429128:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973506 is committed 00002000:00010000:3.0:1789760758.429207:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25341952 -> 25346048 00010000:00010000:3.0:1789760758.429222:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05f57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.429242:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05f57 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.429252:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05f57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.429260:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760758.429265:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05f57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.429276:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05f57 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.429296:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05f57 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.429304:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e05f57 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.429514:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493775744:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36841us (38238us total) trans 4294973506 rc 0/0 00000100:00100000:3.0:1789760758.429525:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6217 00000100:00100000:3.0:1789760758.429543:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768663500666 time 768733693587 state 0x2 ref 0 00000100:00100000:0.0:1789760758.437231:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760758.437328:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493775872 00000100:00100000:2.0:1789760758.437346:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493775872 00000100:00100000:2.0:1789760758.437485:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768663500666 time 768733693587 state 0x0 ref 1 00000100:00100000:2.0:1789760758.437499:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768733954003 time 768733693587 state 0x1 ref 1 00000100:00100000:2.0:1789760758.437522:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768733954003 time 768778558444 state 0x0 ref 1 00000100:00100000:2.0:1789760758.437531:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493775872:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760758.437571:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25346048, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:2.0:1789760758.437577:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f5e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.437589:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760758.437592:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.437597:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.437607:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760758.500701:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973507 is committed 00002000:00010000:2.0:1789760758.500816:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25346048 -> 25350144 00010000:00010000:2.0:1789760758.500832:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.500846:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f5e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.500854:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f5e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.500861:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760758.500865:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f5e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.500875:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f5e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.500897:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f5e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.500904:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f5e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.501037:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493775872:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63501us (63808us total) trans 4294973507 rc 0/0 00000100:00100000:2.0:1789760758.501049:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6218 00000100:00100000:2.0:1789760758.501058:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768733954003 time 768778558444 state 0x2 ref 0 00000100:00100000:1.0:1789760758.507585:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760758.507807:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493776000 00000100:00100000:3.0:1789760758.507817:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493776000 00000100:00100000:3.0:1789760758.507878:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768733954003 time 768778558444 state 0x0 ref 1 00000100:00100000:3.0:1789760758.507893:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768778818860 time 768778558444 state 0x1 ref 1 00000100:00100000:3.0:1789760758.507922:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768778818860 time 768848957168 state 0x0 ref 1 00000100:00100000:3.0:1789760758.507935:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493776000:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760758.508021:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25350144, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:3.0:1789760758.508031:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f65 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.508046:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760758.508050:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.508056:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.508069:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760758.562741:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973508 is committed 00002000:00010000:3.0:1789760758.563545:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25350144 -> 25354240 00010000:00010000:3.0:1789760758.563562:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.563575:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f65 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.563589:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f65 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.563596:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760758.563601:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f65 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.563609:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f65 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.563627:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f65 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.563634:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f65 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.563866:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493776000:12345-192.168.203.60@tcp:4:dd.0 Request processed in 55929us (56286us total) trans 4294973508 rc 0/0 00000100:00100000:3.0:1789760758.563877:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6219 00000100:00100000:3.0:1789760758.563887:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768778818860 time 768848957168 state 0x2 ref 0 00000100:00100000:1.0:1789760758.583582:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760758.583689:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493776128 00000100:00100000:3.0:1789760758.583698:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493776128 00000100:00100000:3.0:1789760758.583752:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768778818860 time 768848957168 state 0x0 ref 1 00000100:00100000:3.0:1789760758.583768:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768849217584 time 768848957168 state 0x1 ref 1 00000100:00100000:3.0:1789760758.583801:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768849217584 time 768924836827 state 0x0 ref 1 00000100:00100000:3.0:1789760758.583813:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493776128:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760758.583860:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25354240, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:3.0:1789760758.583870:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f6c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.583887:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760758.583891:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.583898:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.583911:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760758.632226:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25354240 -> 25358336 00010000:00010000:2.0:1789760758.632264:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.632278:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f6c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.632287:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.632296:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760758.632300:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.632324:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f6c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.632348:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f6c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.632364:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e05f6c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760758.632573:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973509 is committed 00000100:00100000:2.0:1789760758.632685:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493776128:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48867us (49106us total) trans 4294973509 rc 0/0 00000100:00100000:2.0:1789760758.632696:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6220 00000100:00100000:2.0:1789760758.632707:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768849217584 time 768924836827 state 0x2 ref 0 00000100:00100000:2.0:1789760758.640394:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760758.640512:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493776256 00000100:00100000:3.0:1789760758.640521:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493776256 00000100:00100000:3.0:1789760758.640577:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768849217584 time 768924836827 state 0x0 ref 1 00000100:00100000:3.0:1789760758.640592:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768925097243 time 768924836827 state 0x1 ref 1 00000100:00100000:3.0:1789760758.640662:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768925097243 time 768981696454 state 0x0 ref 1 00000100:00100000:3.0:1789760758.640675:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493776256:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760758.640742:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25358336, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xc807 00010000:00010000:3.0:1789760758.640751:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05f73 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.640767:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760758.640771:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.640777:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05f73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.640791:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05f73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.650113:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760758.650135:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc48dce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3304:1876699519255936:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760758.650207:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760758.650695:0:6691:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519255936 00000100:00100000:1.0:1789760758.650703:0:6691:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519255936 00000100:00100000:1.0:1789760758.650813:0:6691:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 206 00000100:00100000:1.0:1789760758.650823:0:6691:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620faf14850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519255936:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760758.650850:0:6691:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527761408 pending 1703936 free 3832545280 avail 3792699392 00000100:00100000:1.0:1789760758.650906:0:6691:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620faf14850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519255936:12345-0@lo:13:osp-pre-0-0.0 Request processed in 85us (698us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760758.650913:0:6691:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 206 00000100:00100000:2.0:1789760758.651146:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc48dce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3304:1876699519255936:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1789760758.651310:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789760758.664809:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25358336 -> 25362432 00002000:00010000:3.0:1789760758.664816:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760757 -> 1789760758 00002000:00010000:3.0:1789760758.664819:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760757 -> 1789760758 00010000:00010000:3.0:1789760758.664831:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05f73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.664844:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05f73 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.664853:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05f73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.664861:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760758.664866:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05f73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.664875:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05f73 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.664895:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05f73 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.664902:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e05f73 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.665020:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493776256:12345-192.168.203.60@tcp:4:dd.0 Request processed in 24343us (24630us total) trans 4294973510 rc 0/0 00000100:00100000:3.0:1789760758.665030:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6221 00000100:00100000:3.0:1789760758.665039:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768925097243 time 768981696454 state 0x2 ref 0 00000001:00080000:3.0:1789760758.665075:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973510 is committed 00000100:00100000:1.0:1789760758.673464:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760758.673609:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493776384 00000100:00100000:2.0:1789760758.673619:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493776384 00000100:00100000:2.0:1789760758.673785:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768925097243 time 768981696454 state 0x0 ref 1 00000100:00100000:2.0:1789760758.673804:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768981956870 time 768981696454 state 0x1 ref 1 00000100:00100000:2.0:1789760758.673880:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768981956870 time 769014914944 state 0x0 ref 1 00000100:00100000:2.0:1789760758.673893:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493776384:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760758.673943:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25362432, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:2.0:1789760758.673952:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f7a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.673998:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:2.0:1789760758.674003:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.674009:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.674024:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760758.724524:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973511 is committed 00002000:00010000:2.0:1789760758.724874:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25362432 -> 25366528 00010000:00010000:2.0:1789760758.724891:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.724903:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f7a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.724911:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f7a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.724918:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760758.724923:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f7a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.725180:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f7a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.725202:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f7a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.725210:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f7a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.725603:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493776384:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51705us (52141us total) trans 4294973511 rc 0/0 00000100:00100000:3.0:1789760758.725613:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6222 00000100:00100000:3.0:1789760758.725624:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768981956870 time 769014914944 state 0x2 ref 0 00000100:00100000:3.0:1789760758.733310:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760758.733437:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493776512 00000100:00100000:2.0:1789760758.733448:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493776512 00000100:00100000:2.0:1789760758.733502:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 768981956870 time 769014914944 state 0x0 ref 1 00000100:00100000:2.0:1789760758.733517:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769015175360 time 769014914944 state 0x1 ref 1 00000100:00100000:2.0:1789760758.733547:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769015175360 time 769074582282 state 0x0 ref 1 00000100:00100000:2.0:1789760758.733558:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493776512:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760758.733609:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25366528, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:2.0:1789760758.733618:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f81 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.733634:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:2.0:1789760758.733638:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.733645:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.733659:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760758.803610:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973512 is committed 00002000:00010000:1.0:1789760758.803754:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25366528 -> 25370624 00010000:00010000:1.0:1789760758.803769:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.803783:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f81 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.803792:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.803799:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760758.803804:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.803814:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f81 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.803833:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f81 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.803841:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e05f81 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.803928:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493776512:12345-192.168.203.60@tcp:4:dd.0 Request processed in 70367us (70622us total) trans 4294973512 rc 0/0 00000100:00100000:1.0:1789760758.803938:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6223 00000100:00100000:1.0:1789760758.803947:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769015175360 time 769074582282 state 0x2 ref 0 00000100:00100000:2.0:1789760758.807657:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760758.807748:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493776640 00000100:00100000:0.0:1789760758.807757:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493776640 00000100:00100000:0.0:1789760758.807810:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769015175360 time 769074582282 state 0x0 ref 1 00000100:00100000:0.0:1789760758.807825:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769074842698 time 769074582282 state 0x1 ref 1 00000100:00100000:0.0:1789760758.807854:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769074842698 time 769148889588 state 0x0 ref 1 00000100:00100000:0.0:1789760758.807866:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493776640:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760758.807914:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25370624, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:0.0:1789760758.807922:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05f88 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.807938:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760758.807943:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.807949:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05f88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.807999:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05f88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760758.887922:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973513 is committed 00002000:00010000:1.0:1789760758.888119:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25370624 -> 25374720 00010000:00010000:1.0:1789760758.888134:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05f88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.888147:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05f88 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.888156:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05f88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.888163:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760758.888168:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05f88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.888176:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05f88 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.888195:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05f88 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.888202:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e05f88 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.888300:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493776640:12345-192.168.203.60@tcp:4:dd.0 Request processed in 80432us (80649us total) trans 4294973513 rc 0/0 00000100:00100000:1.0:1789760758.888311:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6224 00000100:00100000:1.0:1789760758.888321:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769074842698 time 769148889588 state 0x2 ref 0 00000100:00100000:3.0:1789760758.894914:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760758.895593:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493776768 00000100:00100000:0.0:1789760758.895603:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493776768 00000100:00100000:0.0:1789760758.895655:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769074842698 time 769148889588 state 0x0 ref 1 00000100:00100000:0.0:1789760758.895671:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769149150004 time 769148889588 state 0x1 ref 1 00000100:00100000:0.0:1789760758.895756:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769149150004 time 769236791257 state 0x0 ref 1 00000100:00100000:0.0:1789760758.895769:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493776768:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760758.895819:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25374720, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:0.0:1789760758.895828:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05f8f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.895845:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760758.895850:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.895857:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05f8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.895871:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05f8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760758.929041:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25374720 -> 25378816 00010000:00010000:0.0:1789760758.929058:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05f8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.929070:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05f8f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.929080:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05f8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.929087:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760758.929091:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05f8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.929099:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05f8f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.929116:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05f8f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.929124:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05f8f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760758.929217:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493776768:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33446us (34308us total) trans 4294973514 rc 0/0 00000100:00100000:0.0:1789760758.929230:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6225 00000100:00100000:0.0:1789760758.929239:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769149150004 time 769236791257 state 0x2 ref 0 00000001:00080000:2.0:1789760758.929715:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973514 is committed 00000100:00100000:3.0:1789760758.932202:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760758.932697:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493776896 00000100:00100000:0.0:1789760758.932708:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493776896 00000100:00100000:0.0:1789760758.932761:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769149150004 time 769236791257 state 0x0 ref 1 00000100:00100000:0.0:1789760758.932775:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769237051673 time 769236791257 state 0x1 ref 1 00000100:00100000:0.0:1789760758.932796:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769237051673 time 769273832187 state 0x0 ref 1 00000100:00100000:0.0:1789760758.932808:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493776896:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760758.932855:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25378816, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:0.0:1789760758.932863:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05f96 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.932876:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760758.932881:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.932887:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05f96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.932901:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05f96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760758.957837:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973515 is committed 00002000:00010000:1.0:1789760758.958605:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25378816 -> 25382912 00010000:00010000:1.0:1789760758.958620:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05f96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.958632:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05f96 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.958641:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05f96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.958654:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760758.958659:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05f96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.958668:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05f96 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.958687:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05f96 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.958694:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e05f96 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.958825:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493776896:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26014us (26624us total) trans 4294973515 rc 0/0 00000100:00100000:1.0:1789760758.958835:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6226 00000100:00100000:1.0:1789760758.958845:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769237051673 time 769273832187 state 0x2 ref 0 00000100:00100000:1.0:1789760758.962258:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760758.962373:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493777024 00000100:00100000:2.0:1789760758.962384:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493777024 00000100:00100000:2.0:1789760758.962529:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769237051673 time 769273832187 state 0x0 ref 1 00000100:00100000:2.0:1789760758.962544:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769274092603 time 769273832187 state 0x1 ref 1 00000100:00100000:2.0:1789760758.962572:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769274092603 time 769303609062 state 0x0 ref 1 00000100:00100000:2.0:1789760758.962585:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493777024:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760758.962629:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25382912, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:2.0:1789760758.962638:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05f9d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.962652:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:2.0:1789760758.962656:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.962662:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05f9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.962675:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05f9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760759.006844:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973516 is committed 00002000:00010000:2.0:1789760759.007056:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25382912 -> 25387008 00010000:00010000:2.0:1789760759.007071:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05f9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.007084:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05f9d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.007094:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05f9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.007101:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760759.007106:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05f9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.007115:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05f9d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.007135:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05f9d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.007144:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e05f9d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.007244:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493777024:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44657us (44991us total) trans 4294973516 rc 0/0 00000100:00100000:2.0:1789760759.007259:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6227 00000100:00100000:2.0:1789760759.007269:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769274092603 time 769303609062 state 0x2 ref 0 00000100:00100000:3.0:1789760759.010470:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760759.010569:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493777152 00000100:00100000:1.0:1789760759.010579:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493777152 00000100:00100000:1.0:1789760759.010649:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769274092603 time 769303609062 state 0x0 ref 1 00000100:00100000:1.0:1789760759.010665:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769303869478 time 769303609062 state 0x1 ref 1 00000100:00100000:1.0:1789760759.010699:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769303869478 time 769351734805 state 0x0 ref 1 00000100:00100000:1.0:1789760759.010713:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493777152:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760759.010763:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25387008, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:1.0:1789760759.010772:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05fa4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.010787:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760759.010792:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.010798:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05fa4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.010813:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05fa4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760759.011379:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760759.045678:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973517 is committed 00002000:00010000:1.0:1789760759.045786:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25387008 -> 25391104 00010000:00010000:1.0:1789760759.045804:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05fa4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.045817:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05fa4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.045837:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05fa4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.045846:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760759.045859:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05fa4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.045870:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05fa4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.045891:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05fa4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.046058:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05fa4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.046141:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493777152:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35427us (35741us total) trans 4294973517 rc 0/0 00000100:00100000:1.0:1789760759.046688:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6228 00000100:00100000:1.0:1789760759.046698:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769303869478 time 769351734805 state 0x2 ref 0 00000100:00100000:3.0:1789760759.051681:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760759.054030:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493777280 00000100:00100000:0.0:1789760759.054044:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493777280 00000100:00100000:1.0:1789760759.054201:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769303869478 time 769351734805 state 0x0 ref 1 00000100:00100000:1.0:1789760759.054217:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769351995221 time 769351734805 state 0x1 ref 1 00000100:00100000:1.0:1789760759.054403:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769351995221 time 769395436646 state 0x0 ref 1 00000100:00100000:1.0:1789760759.054418:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493777280:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760759.054469:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25391104, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:1.0:1789760759.054477:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05fab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.054493:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760759.054497:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.054504:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05fab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.054518:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05fab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760759.101708:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973518 is committed 00002000:00010000:3.0:1789760759.101940:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25391104 -> 25395200 00010000:00010000:3.0:1789760759.101956:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05fab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.102006:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05fab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.102016:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05fab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.102024:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760759.102030:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05fab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.102039:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05fab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.102060:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05fab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.102068:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e05fab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.102189:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493777280:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47769us (50511us total) trans 4294973518 rc 0/0 00000100:00100000:3.0:1789760759.102201:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6229 00000100:00100000:3.0:1789760759.102212:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769351995221 time 769395436646 state 0x2 ref 0 00000100:00100000:2.0:1789760759.105124:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760759.105254:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493777408 00000100:00100000:1.0:1789760759.105264:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493777408 00000100:00100000:1.0:1789760759.105314:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769351995221 time 769395436646 state 0x0 ref 1 00000100:00100000:1.0:1789760759.105326:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769395697062 time 769395436646 state 0x1 ref 1 00000100:00100000:1.0:1789760759.105363:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769395697062 time 769446398154 state 0x0 ref 1 00000100:00100000:1.0:1789760759.105378:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493777408:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760759.105493:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25395200, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:1.0:1789760759.105503:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05fb2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.105592:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760759.105597:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.105603:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05fb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.105622:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05fb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.139511:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973519 is committed 00002000:00010000:2.0:1789760759.139673:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25395200 -> 25399296 00010000:00010000:2.0:1789760759.139688:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05fb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.139702:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05fb2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.139711:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05fb2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.139719:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760759.139723:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05fb2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.139732:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05fb2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.139752:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05fb2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.139759:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e05fb2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.139877:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493777408:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34497us (34758us total) trans 4294973519 rc 0/0 00000100:00100000:2.0:1789760759.139887:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6230 00000100:00100000:2.0:1789760759.139896:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769395697062 time 769446398154 state 0x2 ref 0 00000100:00100000:2.0:1789760759.145043:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760759.145145:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493777536 00000100:00100000:1.0:1789760759.145154:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493777536 00000100:00100000:1.0:1789760759.145205:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769395697062 time 769446398154 state 0x0 ref 1 00000100:00100000:1.0:1789760759.145218:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769446658570 time 769446398154 state 0x1 ref 1 00000100:00100000:1.0:1789760759.145244:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769446658570 time 769486280260 state 0x0 ref 1 00000100:00100000:1.0:1789760759.145253:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493777536:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760759.145298:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25399296, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:1.0:1789760759.145306:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fb9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.145320:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760759.145323:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.145329:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fb9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.145341:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fb9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.164592:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc5837b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3303:1876699519256448:0@lo:400:kworker.0 00000100:00100000:1.0:1789760759.164662:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760759.164720:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1789760759.164727:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc5835f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3303:1876699519256064:0@lo:400:kworker.0 00000100:00100000:1.0:1789760759.164755:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760759.164779:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620c89fb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3303:1876699519256576:0@lo:400:kworker.0 00000100:00100000:1.0:1789760759.164801:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760759.164838:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc5837800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3305:1876699519256192:0@lo:400:kworker.0 00000100:00100000:1.0:1789760759.164863:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760759.164881:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620c3094700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519256704:0@lo:400:kworker.0 00000100:00100000:1.0:1789760759.164904:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760759.164923:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760759.164928:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc5835c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519256320:0@lo:400:kworker.0 00000100:00100000:1.0:1789760759.164951:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760759.165000:0:10519:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519256448 00000100:00100000:1.0:1789760759.165007:0:10519:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519256448 00000100:00100000:1.0:1789760759.165059:0:10519:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519256576 00000100:00100000:1.0:1789760759.165063:0:10519:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519256576 00000100:00100000:1.0:1789760759.165083:0:10519:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519256192 00000100:00100000:1.0:1789760759.165086:0:10519:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519256192 00000100:00100000:1.0:1789760759.165103:0:10519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 499 00000100:00100000:1.0:1789760759.165111:0:10519:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fab2d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876699519256448:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760759.165164:0:10519:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fab2d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876699519256448:12345-0@lo:400:kworker.0 Request processed in 53us (501us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760759.165172:0:10519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 499 00000100:00100000:1.0:1789760759.165188:0:10519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 500 00000100:00100000:1.0:1789760759.165195:0:10519:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb695880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876699519256576:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760759.165225:0:10519:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb695880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876699519256576:12345-0@lo:400:kworker.0 Request processed in 31us (423us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760759.165231:0:10519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 500 00000100:00100000:1.0:1789760759.165242:0:10519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 501 00000100:00100000:1.0:1789760759.165248:0:10519:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c622f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3305:x1876699519256192:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760759.165276:0:10519:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c622f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3305:x1876699519256192:12345-0@lo:400:kworker.0 Request processed in 28us (412us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760759.165282:0:10519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 501 00000100:00100000:1.0:1789760759.165299:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519256064 00000100:00100000:1.0:1789760759.165302:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519256064 00000100:00100000:1.0:1789760759.165326:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 406 00000100:00100000:1.0:1789760759.165333:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d49050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3303:x1876699519256064:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760759.165372:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d49050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3303:x1876699519256064:12345-0@lo:400:kworker.0 Request processed in 39us (615us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760759.165378:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 406 00000100:00100000:1.0:1789760759.165412:0:10475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519256704 00000100:00100000:1.0:1789760759.165415:0:10475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519256704 00000100:00100000:1.0:1789760759.165439:0:10475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519256320 00000100:00100000:1.0:1789760759.165442:0:10475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519256320 00000100:00100000:1.0:1789760759.165459:0:10475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 180 00000100:00100000:1.0:1789760759.165465:0:10475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fae5ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519256704:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760759.165494:0:10475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fae5ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519256704:12345-0@lo:400:kworker.0 Request processed in 29us (588us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760759.165499:0:10475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 180 00000100:00100000:1.0:1789760759.165510:0:10475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 181 00000100:00100000:1.0:1789760759.165516:0:10475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519256320:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760759.165545:0:10475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519256320:12345-0@lo:400:kworker.0 Request processed in 29us (593us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760759.165551:0:10475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 181 00000100:00080000:1.0:1789760759.165570:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789760759.165574:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760759.165615:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc5835c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519256320:0@lo:400:kworker.0 00000100:00100000:1.0:1789760759.165626:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc5837800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3305:1876699519256192:0@lo:400:kworker.0 00000100:00100000:1.0:1789760759.165634:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620c3094700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519256704:0@lo:400:kworker.0 00000100:00100000:1.0:1789760759.165662:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc5835f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3303:1876699519256064:0@lo:400:kworker.0 00000100:00100000:1.0:1789760759.165672:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620c89fb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3303:1876699519256576:0@lo:400:kworker.0 00000100:00100000:1.0:1789760759.165679:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc5837b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3303:1876699519256448:0@lo:400:kworker.0 00000001:00080000:3.0:1789760759.173786:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973520 is committed 00002000:00010000:1.0:1789760759.174018:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25399296 -> 25403392 00010000:00010000:1.0:1789760759.174032:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fb9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.174044:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fb9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.174053:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fb9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.174060:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760759.174064:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fb9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.174078:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fb9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.174096:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fb9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.174103:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fb9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.174192:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493777536:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28934us (29153us total) trans 4294973520 rc 0/0 00000100:00100000:1.0:1789760759.174203:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6231 00000100:00100000:1.0:1789760759.174211:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769446658570 time 769486280260 state 0x2 ref 0 00000100:00100000:3.0:1789760759.177223:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760759.177328:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493777664 00000100:00100000:0.0:1789760759.177337:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493777664 00000100:00100000:0.0:1789760759.177403:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769446658570 time 769486280260 state 0x0 ref 1 00000100:00100000:0.0:1789760759.177418:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769486540676 time 769486280260 state 0x1 ref 1 00000100:00100000:0.0:1789760759.177449:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769486540676 time 769518484989 state 0x0 ref 1 00000100:00100000:0.0:1789760759.177461:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493777664:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760759.177510:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25403392, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:0.0:1789760759.177519:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05fc0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.177543:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760759.177548:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.177554:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05fc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.177567:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05fc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760759.205436:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973521 is committed 00002000:00010000:0.0:1789760759.205577:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25403392 -> 25407488 00010000:00010000:0.0:1789760759.205590:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05fc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.205603:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05fc0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.205612:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05fc0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.205619:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760759.205623:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05fc0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.205631:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05fc0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.205648:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05fc0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.205655:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05fc0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.205741:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493777664:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28277us (28522us total) trans 4294973521 rc 0/0 00000100:00100000:0.0:1789760759.205751:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6232 00000100:00100000:0.0:1789760759.205761:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769486540676 time 769518484989 state 0x2 ref 0 00000100:00100000:3.0:1789760759.211356:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760759.212379:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493777792 00000100:00100000:1.0:1789760759.212390:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493777792 00000100:00100000:1.0:1789760759.212505:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769486540676 time 769518484989 state 0x0 ref 1 00000100:00100000:1.0:1789760759.212522:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769518745405 time 769518484989 state 0x1 ref 1 00000100:00100000:1.0:1789760759.212554:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769518745405 time 769553590346 state 0x0 ref 1 00000100:00100000:1.0:1789760759.212566:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493777792:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760759.212611:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25407488, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:1.0:1789760759.212620:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fc7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.212634:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760759.212638:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.212644:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.212657:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760759.252000:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973522 is committed 00002000:00010000:1.0:1789760759.252127:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25407488 -> 25411584 00010000:00010000:1.0:1789760759.252142:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.252155:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fc7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.252165:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fc7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.252173:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760759.252177:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fc7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.252187:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fc7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.252208:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fc7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.252216:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fc7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.252305:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493777792:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39736us (40962us total) trans 4294973522 rc 0/0 00000100:00100000:1.0:1789760759.252316:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6233 00000100:00100000:1.0:1789760759.252326:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769518745405 time 769553590346 state 0x2 ref 0 00000100:00100000:2.0:1789760759.257776:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760759.257918:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493777920 00000100:00100000:1.0:1789760759.257929:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493777920 00000100:00100000:1.0:1789760759.258068:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769518745405 time 769553590346 state 0x0 ref 1 00000100:00100000:1.0:1789760759.258083:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769553850762 time 769553590346 state 0x1 ref 1 00000100:00100000:1.0:1789760759.258148:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769553850762 time 769599183377 state 0x0 ref 1 00000100:00100000:1.0:1789760759.258162:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493777920:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760759.258215:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25411584, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:1.0:1789760759.258225:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.258241:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760759.258246:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.258252:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.258272:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760759.287509:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973523 is committed 00002000:00010000:1.0:1789760759.287525:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25411584 -> 25415680 00010000:00010000:1.0:1789760759.287541:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.287554:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.287564:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.287572:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760759.287576:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.287585:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.287605:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.287612:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05fce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.288600:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493777920:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29545us (29938us total) trans 4294973523 rc 0/0 00000100:00100000:1.0:1789760759.288612:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6234 00000100:00100000:1.0:1789760759.288622:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769553850762 time 769599183377 state 0x2 ref 0 00000100:00100000:2.0:1789760759.290384:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760759.290467:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493778048 00000100:00100000:1.0:1789760759.290477:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493778048 00000100:00100000:1.0:1789760759.290535:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769553850762 time 769599183377 state 0x0 ref 1 00000100:00100000:1.0:1789760759.290550:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769599443793 time 769599183377 state 0x1 ref 1 00000100:00100000:1.0:1789760759.290579:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769599443793 time 769631614252 state 0x0 ref 1 00000100:00100000:1.0:1789760759.290591:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493778048:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760759.290637:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25415680, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:1.0:1789760759.290646:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05fd5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.290661:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760759.290665:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.290672:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05fd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.290686:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05fd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760759.328054:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973524 is committed 00002000:00010000:1.0:1789760759.328143:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25415680 -> 25419776 00010000:00010000:1.0:1789760759.328157:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05fd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.328168:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05fd5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.328175:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05fd5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.328182:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760759.328186:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05fd5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.328194:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05fd5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.328211:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05fd5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.328230:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05fd5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.328311:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493778048:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37718us (37932us total) trans 4294973524 rc 0/0 00000100:00100000:1.0:1789760759.328324:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6235 00000100:00100000:1.0:1789760759.328333:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769599443793 time 769631614252 state 0x2 ref 0 00000100:00100000:3.0:1789760759.332860:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760759.332991:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493778176 00000100:00100000:0.0:1789760759.333001:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493778176 00000100:00100000:0.0:1789760759.333055:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769599443793 time 769631614252 state 0x0 ref 1 00000100:00100000:0.0:1789760759.333072:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769631874668 time 769631614252 state 0x1 ref 1 00000100:00100000:0.0:1789760759.333095:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769631874668 time 769674130801 state 0x0 ref 1 00000100:00100000:0.0:1789760759.333106:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493778176:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760759.333150:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25419776, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:0.0:1789760759.333158:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05fdc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.333174:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760759.333178:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.333189:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05fdc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.333204:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05fdc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.357357:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760759.375082:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973525 is committed 00002000:00010000:1.0:1789760759.375233:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25419776 -> 25423872 00010000:00010000:1.0:1789760759.375249:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05fdc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.375261:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05fdc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.375269:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05fdc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.375276:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760759.375281:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05fdc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.375290:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05fdc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.375308:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05fdc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.375315:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e05fdc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.375429:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493778176:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42320us (42573us total) trans 4294973525 rc 0/0 00000100:00100000:1.0:1789760759.375439:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6236 00000100:00100000:1.0:1789760759.375457:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769631874668 time 769674130801 state 0x2 ref 0 00000100:00100000:2.0:1789760759.380560:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760759.381539:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493778304 00000100:00100000:3.0:1789760759.381551:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493778304 00000100:00100000:3.0:1789760759.381658:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769631874668 time 769674130801 state 0x0 ref 1 00000100:00100000:3.0:1789760759.381674:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769674391217 time 769674130801 state 0x1 ref 1 00000100:00100000:3.0:1789760759.381700:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769674391217 time 769722735767 state 0x0 ref 1 00000100:00100000:3.0:1789760759.381712:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493778304:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760759.381865:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25423872, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:3.0:1789760759.381874:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05fe3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.381890:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:3.0:1789760759.381894:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.381909:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05fe3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.381922:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05fe3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760759.414866:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973526 is committed 00002000:00010000:0.0:1789760759.415223:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25423872 -> 25427968 00010000:00010000:0.0:1789760759.415238:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05fe3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.415251:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05fe3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.415259:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05fe3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.415273:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760759.415278:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05fe3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.415288:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05fe3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.415308:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05fe3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.415317:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e05fe3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.415424:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493778304:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33707us (34868us total) trans 4294973526 rc 0/0 00000100:00100000:0.0:1789760759.415434:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6237 00000100:00100000:0.0:1789760759.415445:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769674391217 time 769722735767 state 0x2 ref 0 00000100:00100000:3.0:1789760759.420393:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760759.420531:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493778432 00000100:00100000:1.0:1789760759.420543:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493778432 00000100:00100000:1.0:1789760759.420601:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769674391217 time 769722735767 state 0x0 ref 1 00000100:00100000:1.0:1789760759.420617:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769722996183 time 769722735767 state 0x1 ref 1 00000100:00100000:1.0:1789760759.420682:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769722996183 time 769761717640 state 0x0 ref 1 00000100:00100000:1.0:1789760759.420695:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493778432:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760759.420743:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25427968, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:1.0:1789760759.420752:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05fea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.420768:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760759.420773:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.420779:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05fea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.420793:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05fea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760759.465521:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25427968 -> 25432064 00010000:00010000:3.0:1789760759.465539:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05fea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.465553:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05fea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.465562:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05fea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.465570:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760759.465575:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05fea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.465584:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05fea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.465602:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05fea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.465616:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e05fea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.465705:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493778432:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45007us (45323us total) trans 4294973527 rc 0/0 00000100:00100000:3.0:1789760759.465716:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6238 00000100:00100000:3.0:1789760759.465726:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769722996183 time 769761717640 state 0x2 ref 0 00000001:00080000:0.0:1789760759.466074:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973527 is committed 00000100:00100000:2.0:1789760759.470338:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760759.470690:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493778560 00000100:00100000:1.0:1789760759.470701:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493778560 00000100:00100000:1.0:1789760759.470754:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769722996183 time 769761717640 state 0x0 ref 1 00000100:00100000:1.0:1789760759.470769:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769761978056 time 769761717640 state 0x1 ref 1 00000100:00100000:1.0:1789760759.470801:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769761978056 time 769811836754 state 0x0 ref 1 00000100:00100000:1.0:1789760759.470813:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb07050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493778560:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760759.470860:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25432064, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:1.0:1789760759.470868:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ff1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.470883:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760759.470886:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.470892:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ff1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.470905:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ff1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760759.506939:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973528 is committed 00002000:00010000:3.0:1789760759.507119:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25432064 -> 25436160 00010000:00010000:3.0:1789760759.507134:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ff1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.507147:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ff1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.507157:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ff1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.507164:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760759.507169:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ff1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.507178:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ff1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.507203:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ff1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.507210:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e05ff1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.507304:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb07050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493778560:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36488us (36970us total) trans 4294973528 rc 0/0 00000100:00100000:3.0:1789760759.507314:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6239 00000100:00100000:3.0:1789760759.507324:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769761978056 time 769811836754 state 0x2 ref 0 00000100:00100000:2.0:1789760759.511609:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760759.511895:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493778688 00000100:00100000:0.0:1789760759.511914:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493778688 00000100:00100000:0.0:1789760759.512180:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769761978056 time 769811836754 state 0x0 ref 1 00000100:00100000:0.0:1789760759.512195:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769812097170 time 769811836754 state 0x1 ref 1 00000100:00100000:0.0:1789760759.512224:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769812097170 time 769853260127 state 0x0 ref 1 00000100:00100000:0.0:1789760759.512239:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493778688:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760759.512287:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25436160, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:0.0:1789760759.512296:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05ff8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.512313:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760759.512317:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.512323:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05ff8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.512338:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05ff8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760759.537691:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973529 is committed 00002000:00010000:0.0:1789760759.537777:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25436160 -> 25440256 00010000:00010000:0.0:1789760759.537792:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05ff8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.537804:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05ff8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.537811:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05ff8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.537818:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760759.537823:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05ff8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.537833:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05ff8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.537852:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05ff8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.537860:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e05ff8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.538132:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493778688:12345-192.168.203.60@tcp:4:dd.0 Request processed in 25892us (26528us total) trans 4294973529 rc 0/0 00000100:00100000:0.0:1789760759.538143:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6240 00000100:00100000:0.0:1789760759.538152:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769812097170 time 769853260127 state 0x2 ref 0 00000100:00100000:2.0:1789760759.542162:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760759.542243:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493778816 00000100:00100000:0.0:1789760759.542252:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493778816 00000100:00100000:0.0:1789760759.542312:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769812097170 time 769853260127 state 0x0 ref 1 00000100:00100000:0.0:1789760759.542328:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769853520543 time 769853260127 state 0x1 ref 1 00000100:00100000:0.0:1789760759.542364:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769853520543 time 769883399254 state 0x0 ref 1 00000100:00100000:0.0:1789760759.542378:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493778816:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760759.542436:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25440256, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:0.0:1789760759.542446:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05fff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.542462:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760759.542467:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.542475:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05fff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.542489:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05fff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760759.571318:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25440256 -> 25444352 00010000:00010000:1.0:1789760759.571341:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05fff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.571369:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05fff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.571378:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05fff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.571386:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760759.571390:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05fff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.571464:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05fff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.571484:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05fff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.571492:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e05fff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.571594:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493778816:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29213us (29436us total) trans 4294973530 rc 0/0 00000100:00100000:1.0:1789760759.571605:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6241 00000100:00100000:1.0:1789760759.571614:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769853520543 time 769883399254 state 0x2 ref 0 00000001:00080000:2.0:1789760759.572358:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973530 is committed 00000100:00100000:2.0:1789760759.579158:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760759.579435:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493778944 00000100:00100000:3.0:1789760759.579446:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493778944 00000100:00100000:3.0:1789760759.579515:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769853520543 time 769883399254 state 0x0 ref 1 00000100:00100000:3.0:1789760759.579532:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769883659670 time 769883399254 state 0x1 ref 1 00000100:00100000:3.0:1789760759.579570:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769883659670 time 769920605751 state 0x0 ref 1 00000100:00100000:3.0:1789760759.579584:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493778944:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760759.579634:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25444352, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xc807 00010000:00010000:3.0:1789760759.579643:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06006 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.579661:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:3.0:1789760759.579667:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.579673:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06006 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.579687:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06006 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760759.623695:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25444352 -> 25448448 00002000:00010000:1.0:1789760759.623704:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760758 -> 1789760759 00002000:00010000:1.0:1789760759.623707:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760758 -> 1789760759 00010000:00010000:1.0:1789760759.623720:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06006 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760759.623731:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973531 is committed 00010000:00010000:1.0:1789760759.623733:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06006 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.623745:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06006 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.623752:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760759.623758:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06006 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.623767:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06006 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.623785:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06006 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.623792:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06006 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.625520:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493778944:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45930us (46363us total) trans 4294973531 rc 0/0 00000100:00100000:1.0:1789760759.625534:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6242 00000100:00100000:1.0:1789760759.625545:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769883659670 time 769920605751 state 0x2 ref 0 00000100:00100000:3.0:1789760759.631065:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760759.631160:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493779072 00000100:00100000:1.0:1789760759.631170:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493779072 00000100:00100000:1.0:1789760759.631225:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769883659670 time 769920605751 state 0x0 ref 1 00000100:00100000:1.0:1789760759.631240:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769920866167 time 769920605751 state 0x1 ref 1 00000100:00100000:1.0:1789760759.631271:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769920866167 time 769972305916 state 0x0 ref 1 00000100:00100000:1.0:1789760759.631283:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493779072:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760759.631332:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25448448, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:1.0:1789760759.631354:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0600d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.631371:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760759.631375:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.631382:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0600d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.631397:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0600d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760759.672085:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973532 is committed 00002000:00010000:3.0:1789760759.672254:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25448448 -> 25452544 00010000:00010000:3.0:1789760759.672270:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0600d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.672284:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0600d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.672293:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0600d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.672301:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760759.672306:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0600d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.672315:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0600d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.672336:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0600d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.672362:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0600d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.672465:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493779072:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41179us (41403us total) trans 4294973532 rc 0/0 00000100:00100000:3.0:1789760759.672476:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6243 00000100:00100000:3.0:1789760759.672486:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769920866167 time 769972305916 state 0x2 ref 0 00000100:00100000:3.0:1789760759.677175:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760759.677353:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4ba6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519256832:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760759.677413:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760759.677483:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519256832 00000100:00100000:1.0:1789760759.678064:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519256832 00000100:00100000:1.0:1789760759.678124:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 207 00000100:00100000:1.0:1789760759.678134:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519256832:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760759.678174:0:6690:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760759.678242:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519256832:12345-0@lo:13:osp-pre-1-0.0 Request processed in 109us (828us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760759.678250:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 207 00000100:00100000:3.0:1789760759.678388:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4ba6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519256832:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760759.678594:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493779200 00000100:00100000:0.0:1789760759.678600:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493779200 00000100:00100000:0.0:1789760759.678637:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769920866167 time 769972305916 state 0x0 ref 1 00000100:00100000:0.0:1789760759.678651:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769972566332 time 769972305916 state 0x1 ref 1 00000100:00100000:0.0:1789760759.678675:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769972566332 time 770019710773 state 0x0 ref 1 00000100:00100000:0.0:1789760759.678686:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493779200:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760759.678750:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25452544, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:0.0:1789760759.678759:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06014 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.678773:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760759.678777:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.678783:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06014 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.678796:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06014 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760759.717296:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25452544 -> 25456640 00010000:00010000:2.0:1789760759.717316:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06014 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.717335:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06014 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.717353:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06014 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.717361:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760759.717366:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06014 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.717375:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06014 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.717405:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06014 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.717414:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06014 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.717503:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493779200:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38814us (40334us total) trans 4294973533 rc 0/0 00000100:00100000:2.0:1789760759.717513:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6244 00000100:00100000:2.0:1789760759.717523:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769972566332 time 770019710773 state 0x2 ref 0 00000001:00080000:0.0:1789760759.718339:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973533 is committed 00000100:00100000:0.0:1789760759.721535:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760759.721684:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493779328 00000100:00100000:1.0:1789760759.721694:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493779328 00000100:00100000:1.0:1789760759.721747:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 769972566332 time 770019710773 state 0x0 ref 1 00000100:00100000:1.0:1789760759.721761:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770019971189 time 770019710773 state 0x1 ref 1 00000100:00100000:1.0:1789760759.721827:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770019971189 time 770062861801 state 0x0 ref 1 00000100:00100000:1.0:1789760759.721841:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493779328:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760759.721891:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25456640, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:1.0:1789760759.721899:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0601b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.721916:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:1.0:1789760759.721920:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.721927:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0601b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.721941:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0601b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.761510:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973534 is committed 00002000:00010000:0.0:1789760759.761526:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25456640 -> 25460736 00010000:00010000:0.0:1789760759.761543:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0601b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.761556:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0601b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.761565:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0601b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.761571:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760759.761575:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0601b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.761581:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0601b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.761595:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0601b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.761599:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0601b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.761725:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493779328:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39882us (40195us total) trans 4294973534 rc 0/0 00000100:00100000:0.0:1789760759.761737:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6245 00000100:00100000:0.0:1789760759.761745:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770019971189 time 770062861801 state 0x2 ref 0 00000100:00100000:3.0:1789760759.767307:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760759.767457:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493779456 00000100:00100000:1.0:1789760759.767468:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493779456 00000100:00100000:1.0:1789760759.767519:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770019971189 time 770062861801 state 0x0 ref 1 00000100:00100000:1.0:1789760759.767535:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770063122217 time 770062861801 state 0x1 ref 1 00000100:00100000:1.0:1789760759.767600:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770063122217 time 770108635862 state 0x0 ref 1 00000100:00100000:1.0:1789760759.767613:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493779456:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760759.767659:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25460736, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:1.0:1789760759.767669:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06022 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.767685:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:1.0:1789760759.767690:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.767696:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06022 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.767710:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06022 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.794026:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973535 is committed 00002000:00010000:1.0:1789760759.794138:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25460736 -> 25464832 00010000:00010000:1.0:1789760759.794151:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06022 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.794169:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06022 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.794177:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06022 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.794184:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760759.794189:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06022 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.794198:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06022 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.794219:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06022 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.794226:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06022 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.794320:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493779456:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26705us (27019us total) trans 4294973535 rc 0/0 00000100:00100000:1.0:1789760759.794330:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6246 00000100:00100000:1.0:1789760759.794349:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770063122217 time 770108635862 state 0x2 ref 0 00000100:00100000:3.0:1789760759.798873:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760759.798957:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493779584 00000100:00100000:1.0:1789760759.799014:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493779584 00000100:00100000:1.0:1789760759.799071:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770063122217 time 770108635862 state 0x0 ref 1 00000100:00100000:1.0:1789760759.799085:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770108896278 time 770108635862 state 0x1 ref 1 00000100:00100000:1.0:1789760759.799107:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770108896278 time 770140142701 state 0x0 ref 1 00000100:00100000:1.0:1789760759.799119:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493779584:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760759.799168:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25464832, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:1.0:1789760759.799177:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06029 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.799192:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:1.0:1789760759.799197:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.799203:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06029 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.799217:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06029 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.825578:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973536 is committed 00002000:00010000:1.0:1789760759.825627:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25464832 -> 25468928 00010000:00010000:1.0:1789760759.825644:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06029 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.825657:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06029 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.825672:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06029 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.825679:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760759.825685:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06029 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.825693:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06029 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.825726:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06029 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.825735:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06029 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.825835:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493779584:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26714us (26967us total) trans 4294973536 rc 0/0 00000100:00100000:1.0:1789760759.825846:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6247 00000100:00100000:1.0:1789760759.826062:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770108896278 time 770140142701 state 0x2 ref 0 00000100:00100000:3.0:1789760759.833813:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760759.833883:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493779712 00000100:00100000:2.0:1789760759.833893:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493779712 00000100:00100000:2.0:1789760759.833954:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770108896278 time 770140142701 state 0x0 ref 1 00000100:00100000:2.0:1789760759.834014:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770140403117 time 770140142701 state 0x1 ref 1 00000100:00100000:3.0:1789760759.834172:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770140403117 time 770175204741 state 0x0 ref 1 00000100:00100000:3.0:1789760759.834187:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493779712:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760759.834239:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25468928, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:3.0:1789760759.834248:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06030 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.834263:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760759.834269:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.834276:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06030 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.834291:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06030 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760759.889352:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25468928 -> 25473024 00010000:00010000:0.0:1789760759.889373:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06030 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.889387:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06030 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.889405:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06030 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.889414:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760759.889419:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06030 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.889428:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06030 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.889448:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06030 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.889462:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06030 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.889556:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493779712:12345-192.168.203.60@tcp:4:dd.0 Request processed in 55367us (55748us total) trans 4294973537 rc 0/0 00000100:00100000:0.0:1789760759.889567:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6248 00000100:00100000:0.0:1789760759.889577:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770140403117 time 770175204741 state 0x2 ref 0 00000001:00080000:1.0:1789760759.890299:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973537 is committed 00000100:00100000:2.0:1789760759.896062:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760759.896137:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493779840 00000100:00100000:0.0:1789760759.896146:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493779840 00000100:00100000:0.0:1789760759.896198:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770140403117 time 770175204741 state 0x0 ref 1 00000100:00100000:0.0:1789760759.896211:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770175465157 time 770175204741 state 0x1 ref 1 00000100:00100000:0.0:1789760759.896241:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770175465157 time 770237277206 state 0x0 ref 1 00000100:00100000:0.0:1789760759.896253:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493779840:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760759.896298:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25473024, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:0.0:1789760759.896306:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06037 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.896322:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760759.896326:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.896332:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06037 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.896361:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06037 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760759.922555:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973538 is committed 00002000:00010000:3.0:1789760759.922891:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25473024 -> 25477120 00010000:00010000:3.0:1789760759.922914:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06037 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.922926:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06037 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.922934:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06037 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.922940:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760759.922944:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06037 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.922952:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06037 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.922996:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06037 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.923005:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06037 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.923440:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493779840:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27179us (27377us total) trans 4294973538 rc 0/0 00000100:00100000:3.0:1789760759.923451:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6249 00000100:00100000:3.0:1789760759.923461:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770175465157 time 770237277206 state 0x2 ref 0 00000100:00100000:3.0:1789760759.928616:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760759.928674:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493779968 00000100:00100000:1.0:1789760759.928741:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493779968 00000100:00100000:1.0:1789760759.928797:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770175465157 time 770237277206 state 0x0 ref 1 00000100:00100000:1.0:1789760759.928813:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770237537622 time 770237277206 state 0x1 ref 1 00000100:00100000:1.0:1789760759.928888:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770237537622 time 770269923247 state 0x0 ref 1 00000100:00100000:1.0:1789760759.928902:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493779968:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760759.928947:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25477120, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:1.0:1789760759.928953:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0603e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.928991:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:1.0:1789760759.928996:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.929002:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0603e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.929019:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0603e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.982802:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973539 is committed 00002000:00010000:1.0:1789760759.982880:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25477120 -> 25481216 00010000:00010000:1.0:1789760759.982899:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0603e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.982913:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0603e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.982921:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0603e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.982929:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760759.982933:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0603e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.982942:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0603e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.982993:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0603e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.983002:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0603e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.983100:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493779968:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54196us (54487us total) trans 4294973539 rc 0/0 00000100:00100000:1.0:1789760759.983111:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6250 00000100:00100000:1.0:1789760759.983121:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770237537622 time 770269923247 state 0x2 ref 0 00000100:00100000:0.0:1789760759.993651:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760759.993785:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493780096 00000100:00100000:1.0:1789760759.993795:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493780096 00000100:00100000:1.0:1789760759.993858:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770237537622 time 770269923247 state 0x0 ref 1 00000100:00100000:1.0:1789760759.993872:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770270183663 time 770269923247 state 0x1 ref 1 00000100:00100000:1.0:1789760759.993903:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770270183663 time 770334938495 state 0x0 ref 1 00000100:00100000:1.0:1789760759.993919:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493780096:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760759.994002:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25481216, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:1.0:1789760759.994011:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06045 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.994027:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:1.0:1789760759.994032:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.994038:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06045 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.994051:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06045 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760760.030023:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25481216 -> 25485312 00010000:00010000:3.0:1789760760.030043:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06045 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760760.030272:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973540 is committed 00010000:00010000:3.0:1789760760.030316:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06045 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.030328:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06045 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.030337:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760760.030353:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06045 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.030363:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06045 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.030383:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06045 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.030570:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06045 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.030677:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493780096:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36754us (37029us total) trans 4294973540 rc 0/0 00000100:00100000:3.0:1789760760.030698:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6251 00000100:00100000:3.0:1789760760.030707:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770270183663 time 770334938495 state 0x2 ref 0 00000100:00100000:2.0:1789760760.036233:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760760.036370:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493780224 00000100:00100000:0.0:1789760760.036381:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493780224 00000100:00100000:0.0:1789760760.036598:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770270183663 time 770334938495 state 0x0 ref 1 00000100:00100000:0.0:1789760760.036614:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770335198911 time 770334938495 state 0x1 ref 1 00000100:00100000:0.0:1789760760.036646:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770335198911 time 770377682252 state 0x0 ref 1 00000100:00100000:0.0:1789760760.036660:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493780224:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760760.036710:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25485312, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:0.0:1789760760.036719:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0604c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.036735:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760760.036739:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.036745:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0604c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.036759:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0604c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760760.078084:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25485312 -> 25489408 00010000:00010000:3.0:1789760760.078105:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0604c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.078118:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0604c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.078127:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0604c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.078135:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760760.078140:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0604c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760760.078146:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973541 is committed 00010000:00010000:3.0:1789760760.078148:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0604c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.078172:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0604c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.078181:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e0604c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.078272:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493780224:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41606us (42039us total) trans 4294973541 rc 0/0 00000100:00100000:3.0:1789760760.078283:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6252 00000100:00100000:3.0:1789760760.078293:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770335198911 time 770377682252 state 0x2 ref 0 00000100:00100000:2.0:1789760760.081297:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760760.081404:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493780352 00000100:00100000:3.0:1789760760.081413:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493780352 00000100:00100000:3.0:1789760760.081463:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770335198911 time 770377682252 state 0x0 ref 1 00000100:00100000:3.0:1789760760.081477:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770377942668 time 770377682252 state 0x1 ref 1 00000100:00100000:3.0:1789760760.081543:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770377942668 time 770422577488 state 0x0 ref 1 00000100:00100000:3.0:1789760760.081556:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493780352:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760760.081604:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25489408, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:3.0:1789760760.081612:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06053 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.081627:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760760.081632:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.081639:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06053 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.081653:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06053 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760760.110110:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973542 is committed 00002000:00010000:3.0:1789760760.110602:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25489408 -> 25493504 00010000:00010000:3.0:1789760760.110618:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06053 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.110631:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06053 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.110641:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06053 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.110650:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760760.110655:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06053 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.110665:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06053 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.110685:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06053 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.110699:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06053 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.113137:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493780352:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31572us (31837us total) trans 4294973542 rc 0/0 00000100:00100000:3.0:1789760760.113152:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6253 00000100:00100000:3.0:1789760760.113162:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770377942668 time 770422577488 state 0x2 ref 0 00000100:00100000:0.0:1789760760.118463:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760760.118575:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493780480 00000100:00100000:1.0:1789760760.118585:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493780480 00000100:00100000:1.0:1789760760.118639:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770377942668 time 770422577488 state 0x0 ref 1 00000100:00100000:1.0:1789760760.118656:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770422837904 time 770422577488 state 0x1 ref 1 00000100:00100000:1.0:1789760760.118717:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770422837904 time 770459751786 state 0x0 ref 1 00000100:00100000:1.0:1789760760.118730:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493780480:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760760.118781:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25493504, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:1.0:1789760760.118790:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0605a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.118936:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:1.0:1789760760.118941:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.118948:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0605a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.118980:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0605a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760760.163204:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25493504 -> 25497600 00010000:00010000:3.0:1789760760.163224:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0605a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.163238:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0605a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.163247:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0605a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.163254:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760760.163259:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0605a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.163269:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0605a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.163293:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0605a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.163301:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0605a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.163411:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493780480:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44678us (44953us total) trans 4294973543 rc 0/0 00000100:00100000:3.0:1789760760.163422:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6254 00000100:00100000:3.0:1789760760.163432:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770422837904 time 770459751786 state 0x2 ref 0 00000001:00080000:2.0:1789760760.164140:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973543 is committed 00000100:00100000:2.0:1789760760.168597:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760760.168700:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493780608 00000100:00100000:3.0:1789760760.168710:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493780608 00000100:00100000:3.0:1789760760.168764:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770422837904 time 770459751786 state 0x0 ref 1 00000100:00100000:3.0:1789760760.168780:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770460012202 time 770459751786 state 0x1 ref 1 00000100:00100000:3.0:1789760760.169043:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770460012202 time 770510076149 state 0x0 ref 1 00000100:00100000:3.0:1789760760.169057:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493780608:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760760.169108:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25497600, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:3.0:1789760760.169118:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06061 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.169132:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760760.169136:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.169143:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06061 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.169157:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06061 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760760.214278:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973544 is committed 00002000:00010000:2.0:1789760760.214473:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25497600 -> 25501696 00010000:00010000:2.0:1789760760.214487:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06061 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.214499:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06061 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.214508:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06061 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.214515:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760760.214519:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06061 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.214529:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06061 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.214552:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06061 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.214559:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06061 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.214680:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493780608:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45621us (46087us total) trans 4294973544 rc 0/0 00000100:00100000:2.0:1789760760.214690:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6255 00000100:00100000:2.0:1789760760.214699:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770460012202 time 770510076149 state 0x2 ref 0 00000100:00100000:0.0:1789760760.221091:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760760.221247:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493780736 00000100:00100000:3.0:1789760760.221257:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493780736 00000100:00100000:3.0:1789760760.221309:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770460012202 time 770510076149 state 0x0 ref 1 00000100:00100000:3.0:1789760760.221324:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770510336565 time 770510076149 state 0x1 ref 1 00000100:00100000:3.0:1789760760.221366:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770510336565 time 770562401036 state 0x0 ref 1 00000100:00100000:3.0:1789760760.221379:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493780736:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760760.221427:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25501696, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:3.0:1789760760.221437:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06068 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.221451:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760760.221456:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.221462:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06068 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.221476:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06068 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760760.251176:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973545 is committed 00002000:00010000:2.0:1789760760.251273:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25501696 -> 25505792 00010000:00010000:2.0:1789760760.251288:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06068 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.251302:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06068 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.251311:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06068 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.251320:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760760.251326:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06068 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.251347:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06068 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.251369:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06068 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.251377:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06068 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.251478:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493780736:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30096us (30391us total) trans 4294973545 rc 0/0 00000100:00100000:2.0:1789760760.251489:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6256 00000100:00100000:2.0:1789760760.251499:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770510336565 time 770562401036 state 0x2 ref 0 00000100:00100000:0.0:1789760760.256253:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760760.256535:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493780864 00000100:00100000:3.0:1789760760.256544:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493780864 00000100:00100000:3.0:1789760760.256596:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770510336565 time 770562401036 state 0x0 ref 1 00000100:00100000:3.0:1789760760.256678:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770562661452 time 770562401036 state 0x1 ref 1 00000100:00100000:3.0:1789760760.256742:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770562661452 time 770597777251 state 0x0 ref 1 00000100:00100000:3.0:1789760760.256757:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493780864:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760760.256804:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25505792, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:3.0:1789760760.256813:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0606f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.256828:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760760.256833:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.256839:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0606f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.256853:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0606f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760760.292048:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973546 is committed 00002000:00010000:2.0:1789760760.292407:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25505792 -> 25509888 00010000:00010000:2.0:1789760760.292422:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0606f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.292435:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0606f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.292445:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0606f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.292457:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760760.292462:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0606f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.292472:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0606f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.292492:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0606f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.292500:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0606f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.292594:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493780864:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35835us (36629us total) trans 4294973546 rc 0/0 00000100:00100000:2.0:1789760760.292824:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6257 00000100:00100000:2.0:1789760760.292834:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770562661452 time 770597777251 state 0x2 ref 0 00000100:00100000:1.0:1789760760.296288:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760760.296397:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493780992 00000100:00100000:3.0:1789760760.296465:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493780992 00000100:00100000:3.0:1789760760.296519:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770562661452 time 770597777251 state 0x0 ref 1 00000100:00100000:3.0:1789760760.296535:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770598037667 time 770597777251 state 0x1 ref 1 00000100:00100000:3.0:1789760760.296596:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770598037667 time 770637630893 state 0x0 ref 1 00000100:00100000:3.0:1789760760.296618:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493780992:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760760.296669:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25509888, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:3.0:1789760760.296678:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06076 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.296693:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760760.296697:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.296703:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06076 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.296716:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06076 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760760.324580:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973547 is committed 00002000:00010000:1.0:1789760760.324763:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25509888 -> 25513984 00010000:00010000:1.0:1789760760.324778:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06076 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.324791:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06076 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.324799:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06076 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.324806:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760760.324811:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06076 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.324820:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06076 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.324842:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06076 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.324849:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06076 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.325098:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493780992:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28477us (28814us total) trans 4294973547 rc 0/0 00000100:00100000:1.0:1789760760.325110:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6258 00000100:00100000:1.0:1789760760.325120:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770598037667 time 770637630893 state 0x2 ref 0 00000100:00100000:0.0:1789760760.331559:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760760.332812:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493781120 00000100:00100000:2.0:1789760760.332890:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493781120 00000100:00100000:2.0:1789760760.333004:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770598037667 time 770637630893 state 0x0 ref 1 00000100:00100000:2.0:1789760760.333022:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770637891309 time 770637630893 state 0x1 ref 1 00000100:00100000:2.0:1789760760.333117:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770637891309 time 770674151760 state 0x0 ref 1 00000100:00100000:2.0:1789760760.333135:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493781120:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760760.333187:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25513984, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:2.0:1789760760.333197:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0607d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.333214:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760760.333219:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.333226:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0607d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.333240:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0607d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760760.360623:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973548 is committed 00002000:00010000:2.0:1789760760.360727:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25513984 -> 25518080 00010000:00010000:2.0:1789760760.360743:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0607d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.360756:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0607d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.360765:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0607d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.360773:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760760.360778:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0607d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.360788:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0607d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.360809:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0607d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.360895:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0607d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.361114:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493781120:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27978us (29557us total) trans 4294973548 rc 0/0 00000100:00100000:2.0:1789760760.361125:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6259 00000100:00100000:2.0:1789760760.361135:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770637891309 time 770674151760 state 0x2 ref 0 00000100:00100000:3.0:1789760760.365692:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760760.365920:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493781248 00000100:00100000:2.0:1789760760.365929:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493781248 00000100:00100000:2.0:1789760760.366015:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770637891309 time 770674151760 state 0x0 ref 1 00000100:00100000:2.0:1789760760.366030:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770674412176 time 770674151760 state 0x1 ref 1 00000100:00100000:2.0:1789760760.366053:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770674412176 time 770707089917 state 0x0 ref 1 00000100:00100000:2.0:1789760760.366065:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493781248:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760760.366189:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25518080, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:2.0:1789760760.366198:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06084 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.366213:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760760.366218:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.366224:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06084 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.366236:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06084 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.382301:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760760.408754:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25518080 -> 25522176 00010000:00010000:0.0:1789760760.408776:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06084 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.408789:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06084 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.408799:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06084 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.408806:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760760.408812:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06084 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760760.408821:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973549 is committed 00010000:00010000:0.0:1789760760.408897:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06084 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.408917:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06084 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.408925:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06084 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.409285:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493781248:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43216us (43595us total) trans 4294973549 rc 0/0 00000100:00100000:1.0:1789760760.409297:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6260 00000100:00100000:1.0:1789760760.409306:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770674412176 time 770707089917 state 0x2 ref 0 00000100:00100000:1.0:1789760760.412668:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760760.412753:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493781376 00000100:00100000:3.0:1789760760.412762:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493781376 00000100:00100000:3.0:1789760760.412814:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770674412176 time 770707089917 state 0x0 ref 1 00000100:00100000:3.0:1789760760.412897:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770707350333 time 770707089917 state 0x1 ref 1 00000100:00100000:3.0:1789760760.412931:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770707350333 time 770753966723 state 0x0 ref 1 00000100:00100000:3.0:1789760760.412945:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493781376:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760760.413018:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25522176, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:3.0:1789760760.413027:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0608b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.413042:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760760.413046:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.413052:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0608b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.413066:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0608b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760760.449841:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973550 is committed 00002000:00010000:2.0:1789760760.450164:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25522176 -> 25526272 00010000:00010000:2.0:1789760760.450182:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0608b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.450195:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0608b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.450203:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0608b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.450210:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760760.450214:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0608b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.450223:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0608b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.450242:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0608b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.450251:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0608b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.450351:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493781376:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37396us (37680us total) trans 4294973550 rc 0/0 00000100:00100000:2.0:1789760760.450362:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6261 00000100:00100000:2.0:1789760760.450372:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770707350333 time 770753966723 state 0x2 ref 0 00000100:00100000:1.0:1789760760.456624:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760760.456812:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493781504 00000100:00100000:3.0:1789760760.456832:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493781504 00000100:00100000:3.0:1789760760.456889:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770707350333 time 770753966723 state 0x0 ref 1 00000100:00100000:3.0:1789760760.456905:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770754227139 time 770753966723 state 0x1 ref 1 00000100:00100000:3.0:1789760760.456937:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770754227139 time 770797973110 state 0x0 ref 1 00000100:00100000:3.0:1789760760.456950:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c89fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493781504:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760760.457225:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25526272, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:3.0:1789760760.457235:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06092 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.457252:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760760.457257:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.457263:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06092 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.457277:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06092 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760760.488109:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973551 is committed 00002000:00010000:3.0:1789760760.488265:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25526272 -> 25530368 00010000:00010000:3.0:1789760760.488281:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06092 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.488293:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06092 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.488302:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06092 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.488310:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760760.488315:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06092 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.488324:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06092 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.488355:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06092 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.488363:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06092 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.488464:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c89fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493781504:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31512us (31844us total) trans 4294973551 rc 0/0 00000100:00100000:3.0:1789760760.488476:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6262 00000100:00100000:3.0:1789760760.488485:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770754227139 time 770797973110 state 0x2 ref 0 00000100:00100000:2.0:1789760760.497730:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760760.498037:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493781632 00000100:00100000:3.0:1789760760.498047:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493781632 00000100:00100000:3.0:1789760760.498097:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770754227139 time 770797973110 state 0x0 ref 1 00000100:00100000:3.0:1789760760.498111:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770798233526 time 770797973110 state 0x1 ref 1 00000100:00100000:3.0:1789760760.498139:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770798233526 time 770839175029 state 0x0 ref 1 00000100:00100000:3.0:1789760760.498150:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493781632:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760760.498198:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25530368, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:3.0:1789760760.498216:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06099 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.498233:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760760.498237:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.498243:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06099 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.498258:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06099 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.534052:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973552 is committed 00002000:00010000:3.0:1789760760.534062:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25530368 -> 25534464 00010000:00010000:3.0:1789760760.534078:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06099 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.534091:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06099 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.534167:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06099 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.534175:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760760.534180:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06099 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.534189:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06099 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.534214:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06099 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.534222:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06099 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.534319:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493781632:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36165us (36594us total) trans 4294973552 rc 0/0 00000100:00100000:3.0:1789760760.534333:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6263 00000100:00100000:3.0:1789760760.534351:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770798233526 time 770839175029 state 0x2 ref 0 00000100:00100000:2.0:1789760760.537622:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760760.537774:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493781760 00000100:00100000:1.0:1789760760.537784:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493781760 00000100:00100000:1.0:1789760760.537847:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770798233526 time 770839175029 state 0x0 ref 1 00000100:00100000:1.0:1789760760.537862:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770839435445 time 770839175029 state 0x1 ref 1 00000100:00100000:1.0:1789760760.537896:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770839435445 time 770878931558 state 0x0 ref 1 00000100:00100000:1.0:1789760760.537910:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493781760:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760760.537958:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25534464, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:1.0:1789760760.538004:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e060a0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.538019:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:1.0:1789760760.538024:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.538030:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e060a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.538044:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e060a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.579577:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973553 is committed 00002000:00010000:2.0:1789760760.579757:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25534464 -> 25538560 00010000:00010000:2.0:1789760760.579773:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e060a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.579786:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e060a0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.579794:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e060a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.579801:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760760.579806:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e060a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.579821:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e060a0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.579840:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e060a0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.579847:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e060a0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.579940:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493781760:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42029us (42324us total) trans 4294973553 rc 0/0 00000100:00100000:2.0:1789760760.579950:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6264 00000100:00100000:2.0:1789760760.580000:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770839435445 time 770878931558 state 0x2 ref 0 00000100:00100000:0.0:1789760760.594654:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760760.594776:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493781888 00000100:00100000:2.0:1789760760.594786:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493781888 00000100:00100000:2.0:1789760760.594840:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770839435445 time 770878931558 state 0x0 ref 1 00000100:00100000:2.0:1789760760.594854:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770879191974 time 770878931558 state 0x1 ref 1 00000100:00100000:2.0:1789760760.594947:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770879191974 time 770935980719 state 0x0 ref 1 00000100:00100000:2.0:1789760760.595054:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493781888:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760760.595107:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25538560, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xc807 00010000:00010000:2.0:1789760760.595117:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e060a7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.595134:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760760.595139:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.595145:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e060a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.595160:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e060a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.629235:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973554 is committed 00002000:00010000:2.0:1789760760.629356:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25538560 -> 25542656 00002000:00010000:2.0:1789760760.629361:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760759 -> 1789760760 00002000:00010000:2.0:1789760760.629363:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760759 -> 1789760760 00010000:00010000:2.0:1789760760.629376:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e060a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.629512:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e060a7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.629523:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e060a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.629530:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760760.629535:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e060a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.629543:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e060a7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.629564:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e060a7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.629572:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e060a7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.629710:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493781888:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34748us (35060us total) trans 4294973554 rc 0/0 00000100:00100000:2.0:1789760760.629720:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6265 00000100:00100000:2.0:1789760760.629730:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770879191974 time 770935980719 state 0x2 ref 0 00000100:00100000:3.0:1789760760.633186:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760760.633254:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493782016 00000100:00100000:1.0:1789760760.633263:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493782016 00000100:00100000:1.0:1789760760.633317:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770879191974 time 770935980719 state 0x0 ref 1 00000100:00100000:1.0:1789760760.633333:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770936241135 time 770935980719 state 0x1 ref 1 00000100:00100000:1.0:1789760760.633422:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770936241135 time 770974457121 state 0x0 ref 1 00000100:00100000:1.0:1789760760.633436:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493782016:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760760.633484:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25542656, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc807 00010000:00010000:1.0:1789760760.633493:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e060ae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.633508:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=770s v=(0 0) 00010000:00010000:1.0:1789760760.633513:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.633519:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e060ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.633533:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e060ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760760.663745:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973555 is committed 00002000:00010000:3.0:1789760760.663841:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25542656 -> 25546752 00010000:00010000:3.0:1789760760.663855:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e060ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.663868:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e060ae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.663878:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e060ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.663886:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760760.663891:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e060ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.663900:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e060ae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.664024:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e060ae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.664033:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e060ae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.664185:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493782016:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30747us (31004us total) trans 4294973555 rc 0/0 00000100:00100000:3.0:1789760760.664196:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6266 00000100:00100000:3.0:1789760760.664205:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770936241135 time 770974457121 state 0x2 ref 0 00000100:00100000:0.0:1789760760.668262:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760760.671152:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493782144 00000100:00100000:3.0:1789760760.671167:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493782144 00000100:00100000:3.0:1789760760.671226:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770936241135 time 770974457121 state 0x0 ref 1 00000100:00100000:3.0:1789760760.671243:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770974717537 time 770974457121 state 0x1 ref 1 00000100:00100000:3.0:1789760760.671319:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770974717537 time 771012352964 state 0x0 ref 1 00000100:00100000:3.0:1789760760.671333:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493782144:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760760.671389:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25546752, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc807 00010000:00010000:3.0:1789760760.671399:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e060b5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.671414:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760760.671420:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.671426:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e060b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.671440:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e060b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760760.700316:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25546752 -> 25550848 00010000:00010000:2.0:1789760760.700346:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e060b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.700360:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e060b5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.700369:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e060b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.700377:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760760.700382:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e060b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.700472:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e060b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.700491:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e060b5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.700499:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e060b5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.700606:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493782144:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29271us (32349us total) trans 4294973556 rc 0/0 00000100:00100000:2.0:1789760760.700616:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6267 00000100:00100000:2.0:1789760760.700627:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770974717537 time 771012352964 state 0x2 ref 0 00000001:00080000:2.0:1789760760.700668:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973556 is committed 00000100:00100000:2.0:1789760760.709191:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760760.709737:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493782272 00000100:00100000:1.0:1789760760.709747:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493782272 00000100:00100000:1.0:1789760760.709797:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 770974717537 time 771012352964 state 0x0 ref 1 00000100:00100000:1.0:1789760760.709811:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771012613380 time 771012352964 state 0x1 ref 1 00000100:00100000:1.0:1789760760.709847:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771012613380 time 771050882925 state 0x0 ref 1 00000100:00100000:1.0:1789760760.709859:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493782272:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760760.710157:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25550848, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc807 00010000:00010000:0.0:1789760760.710167:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e060bc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.710183:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760760.710188:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.710194:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e060bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.710208:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e060bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760760.763345:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973557 is committed 00002000:00010000:0.0:1789760760.763539:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25550848 -> 25554944 00010000:00010000:0.0:1789760760.763557:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e060bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.763570:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e060bc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.763579:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e060bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.763591:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760760.763597:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e060bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.763607:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e060bc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.763630:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e060bc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.763638:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e060bc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.764029:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493782272:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54163us (54839us total) trans 4294973557 rc 0/0 00000100:00100000:0.0:1789760760.764043:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6268 00000100:00100000:0.0:1789760760.764055:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771012613380 time 771050882925 state 0x2 ref 0 00000100:00100000:0.0:1789760760.769624:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760760.769794:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493782400 00000100:00100000:2.0:1789760760.769806:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493782400 00000100:00100000:2.0:1789760760.769884:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771012613380 time 771050882925 state 0x0 ref 1 00000100:00100000:2.0:1789760760.770313:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771051143341 time 771050882925 state 0x1 ref 1 00000100:00100000:2.0:1789760760.770361:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771051143341 time 771111395985 state 0x0 ref 1 00000100:00100000:2.0:1789760760.770378:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493782400:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760760.770444:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25554944, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc807 00010000:00010000:2.0:1789760760.770455:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e060c3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.770473:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:2.0:1789760760.770479:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.770486:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e060c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.770502:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e060c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.825602:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973558 is committed 00002000:00010000:3.0:1789760760.825712:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25554944 -> 25559040 00010000:00010000:3.0:1789760760.825731:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e060c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.825746:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e060c3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.825755:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e060c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.825831:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760760.825838:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e060c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.825849:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e060c3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.825873:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e060c3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.825881:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e060c3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760760.832695:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493782400:12345-192.168.203.60@tcp:4:dd.0 Request processed in 62302us (63067us total) trans 4294973558 rc 0/0 00000100:00100000:2.0:1789760760.832715:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6269 00000100:00100000:2.0:1789760760.832727:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771051143341 time 771111395985 state 0x2 ref 0 00000100:00100000:0.0:1789760760.836120:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760760.836353:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493782528 00000100:00100000:3.0:1789760760.836366:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493782528 00000100:00100000:3.0:1789760760.836449:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771051143341 time 771111395985 state 0x0 ref 1 00000100:00100000:3.0:1789760760.836468:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771111656401 time 771111395985 state 0x1 ref 1 00000100:00100000:3.0:1789760760.836494:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771111656401 time 771177528683 state 0x0 ref 1 00000100:00100000:3.0:1789760760.836508:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493782528:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760760.836572:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25559040, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc807 00010000:00010000:3.0:1789760760.836583:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e060ca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.836603:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760760.836609:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.836616:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e060ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.836632:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e060ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760760.870226:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973559 is committed 00002000:00010000:1.0:1789760760.870368:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25559040 -> 25563136 00010000:00010000:1.0:1789760760.870396:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e060ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.870411:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e060ca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.870427:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e060ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.870435:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760760.870441:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e060ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.870451:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e060ca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.870477:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e060ca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.870486:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e060ca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.870605:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493782528:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34093us (34494us total) trans 4294973559 rc 0/0 00000100:00100000:1.0:1789760760.870617:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6270 00000100:00100000:1.0:1789760760.870629:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771111656401 time 771177528683 state 0x2 ref 0 00000100:00100000:3.0:1789760760.877664:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760760.877758:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493782656 00000100:00100000:2.0:1789760760.877786:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493782656 00000100:00100000:2.0:1789760760.877868:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771111656401 time 771177528683 state 0x0 ref 1 00000100:00100000:2.0:1789760760.877887:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771177789099 time 771177528683 state 0x1 ref 1 00000100:00100000:2.0:1789760760.877928:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771177789099 time 771218963103 state 0x0 ref 1 00000100:00100000:2.0:1789760760.877944:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493782656:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760760.878213:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25563136, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc807 00010000:00010000:2.0:1789760760.878225:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e060d1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.878243:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:2.0:1789760760.878249:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.878257:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e060d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.878284:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e060d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760760.926149:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973560 is committed 00002000:00010000:0.0:1789760760.926300:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25563136 -> 25567232 00010000:00010000:0.0:1789760760.926320:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e060d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.926350:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e060d1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.926361:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e060d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.926369:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760760.926375:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e060d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.926395:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e060d1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.926421:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e060d1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.926430:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e060d1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.926563:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493782656:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48616us (48901us total) trans 4294973560 rc 0/0 00000100:00100000:0.0:1789760760.926576:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6271 00000100:00100000:0.0:1789760760.926589:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771177789099 time 771218963103 state 0x2 ref 0 00000100:00100000:0.0:1789760760.934025:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760760.934626:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493782784 00000100:00100000:2.0:1789760760.934643:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493782784 00000100:00100000:2.0:1789760760.934729:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771177789099 time 771218963103 state 0x0 ref 1 00000100:00100000:2.0:1789760760.934747:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771219223519 time 771218963103 state 0x1 ref 1 00000100:00100000:2.0:1789760760.934785:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771219223519 time 771275819898 state 0x0 ref 1 00000100:00100000:2.0:1789760760.934799:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493782784:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760760.934941:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25567232, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc807 00010000:00010000:2.0:1789760760.934951:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e060d8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.935481:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:2.0:1789760760.935488:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.935496:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e060d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.935526:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e060d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760761.027467:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25567232 -> 25571328 00010000:00010000:0.0:1789760761.027496:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e060d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.027796:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e060d8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.027806:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e060d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.027815:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760761.027820:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e060d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.027830:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e060d8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.027855:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e060d8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.027864:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e060d8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760761.028048:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493782784:12345-192.168.203.60@tcp:4:dd.0 Request processed in 93243us (94030us total) trans 4294973561 rc 0/0 00000100:00100000:0.0:1789760761.028061:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6272 00000100:00100000:0.0:1789760761.028074:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771219223519 time 771275819898 state 0x2 ref 0 00000001:00080000:1.0:1789760761.028233:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973561 is committed 00000100:00100000:3.0:1789760761.031236:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760761.031361:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493782912 00000100:00100000:0.0:1789760761.031376:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493782912 00000100:00100000:0.0:1789760761.031480:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771219223519 time 771275819898 state 0x0 ref 1 00000100:00100000:0.0:1789760761.031509:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771276080314 time 771275819898 state 0x1 ref 1 00000100:00100000:0.0:1789760761.031544:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771276080314 time 771372578959 state 0x0 ref 1 00000100:00100000:0.0:1789760761.031560:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493782912:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760761.031626:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25571328, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc807 00010000:00010000:0.0:1789760761.031637:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e060df lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.031820:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760761.031827:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760761.031835:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e060df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.031853:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e060df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760761.033228:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760761.089511:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973562 is committed 00002000:00010000:1.0:1789760761.089791:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25571328 -> 25575424 00010000:00010000:1.0:1789760761.089814:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e060df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.089828:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e060df lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.089837:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e060df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.089844:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760761.089850:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e060df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.089861:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e060df lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.089888:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e060df lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.089896:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e060df lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.090139:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493782912:12345-192.168.203.60@tcp:4:dd.0 Request processed in 58576us (58910us total) trans 4294973562 rc 0/0 00000100:00100000:1.0:1789760761.090152:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6273 00000100:00100000:1.0:1789760761.090165:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771276080314 time 771372578959 state 0x2 ref 0 00000100:00100000:3.0:1789760761.094753:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760761.094928:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493783040 00000100:00100000:1.0:1789760761.094942:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493783040 00000100:00100000:1.0:1789760761.095083:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771276080314 time 771372578959 state 0x0 ref 1 00000100:00100000:1.0:1789760761.095104:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771372839375 time 771372578959 state 0x1 ref 1 00000100:00100000:1.0:1789760761.095132:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771372839375 time 771436167065 state 0x0 ref 1 00000100:00100000:1.0:1789760761.095147:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493783040:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760761.095223:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25575424, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc807 00010000:00010000:1.0:1789760761.095236:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e060e6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.095256:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:1.0:1789760761.095262:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.095270:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e060e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.095287:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e060e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760761.144483:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973563 is committed 00002000:00010000:1.0:1789760761.144589:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25575424 -> 25579520 00010000:00010000:1.0:1789760761.144608:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e060e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.144630:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e060e6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.144640:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e060e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.144648:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760761.144653:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e060e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.144664:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e060e6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.144687:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e060e6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.144696:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e060e6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.144831:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493783040:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49680us (50086us total) trans 4294973563 rc 0/0 00000100:00100000:1.0:1789760761.144853:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6274 00000100:00100000:1.0:1789760761.144864:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771372839375 time 771436167065 state 0x2 ref 0 00000100:00100000:3.0:1789760761.149747:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760761.156548:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493783168 00000100:00100000:2.0:1789760761.156569:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493783168 00000100:00100000:2.0:1789760761.156658:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771372839375 time 771436167065 state 0x0 ref 1 00000100:00100000:2.0:1789760761.156678:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771436427481 time 771436167065 state 0x1 ref 1 00000100:00100000:2.0:1789760761.156715:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771436427481 time 771497749806 state 0x0 ref 1 00000100:00100000:2.0:1789760761.156730:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493783168:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760761.156795:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25579520, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc807 00010000:00010000:2.0:1789760761.156805:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e060ed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.156825:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:2.0:1789760761.156831:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.156839:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e060ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.156856:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e060ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760761.245665:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973564 is committed 00002000:00010000:0.0:1789760761.246661:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25579520 -> 25583616 00010000:00010000:0.0:1789760761.246685:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e060ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.246701:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e060ed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.246713:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e060ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.246723:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760761.246729:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e060ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.246740:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e060ed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.246765:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e060ed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.246773:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e060ed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760761.247396:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493783168:12345-192.168.203.60@tcp:4:dd.0 Request processed in 90660us (97655us total) trans 4294973564 rc 0/0 00000100:00100000:0.0:1789760761.247411:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6275 00000100:00100000:0.0:1789760761.247423:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771436427481 time 771497749806 state 0x2 ref 0 00000100:00100000:0.0:1789760761.265616:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760761.265880:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493783296 00000100:00100000:3.0:1789760761.265894:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493783296 00000100:00100000:3.0:1789760761.266030:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771436427481 time 771497749806 state 0x0 ref 1 00000100:00100000:3.0:1789760761.266052:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771498010222 time 771497749806 state 0x1 ref 1 00000100:00100000:1.0:1789760761.266171:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771498010222 time 771607203184 state 0x0 ref 1 00000100:00100000:1.0:1789760761.266189:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493783296:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760761.266253:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25583616, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc807 00010000:00010000:1.0:1789760761.266263:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e060f4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.266283:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:1.0:1789760761.266289:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.266296:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e060f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.266312:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e060f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760761.340445:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973565 is committed 00002000:00010000:1.0:1789760761.340558:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25583616 -> 25587712 00010000:00010000:1.0:1789760761.340579:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e060f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.340595:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e060f4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.340606:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e060f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.340614:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760761.340621:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e060f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.340632:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e060f4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.340654:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e060f4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.340661:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e060f4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.340773:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493783296:12345-192.168.203.60@tcp:4:dd.0 Request processed in 74583us (75166us total) trans 4294973565 rc 0/0 00000100:00100000:1.0:1789760761.340785:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6276 00000100:00100000:1.0:1789760761.340797:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771498010222 time 771607203184 state 0x2 ref 0 00000100:00100000:1.0:1789760761.344205:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760761.344657:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493783424 00000100:00100000:2.0:1789760761.344671:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493783424 00000100:00100000:2.0:1789760761.344745:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771498010222 time 771607203184 state 0x0 ref 1 00000100:00100000:2.0:1789760761.344764:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771607463600 time 771607203184 state 0x1 ref 1 00000100:00100000:2.0:1789760761.344901:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771607463600 time 771685935254 state 0x0 ref 1 00000100:00100000:2.0:1789760761.344917:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c3094700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493783424:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760761.345018:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25587712, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc807 00010000:00010000:2.0:1789760761.345028:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e060fb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.345047:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:2.0:1789760761.345053:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.345061:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e060fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.345078:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e060fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.410253:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760761.419116:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25587712 -> 25591808 00010000:00010000:1.0:1789760761.419145:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e060fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.419162:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e060fb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.419172:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e060fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.419185:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760761.419190:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e060fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.419200:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e060fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.419885:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e060fb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.419897:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e060fb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.420136:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c3094700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493783424:12345-192.168.203.60@tcp:4:dd.0 Request processed in 75214us (75937us total) trans 4294973566 rc 0/0 00000100:00100000:1.0:1789760761.420149:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6277 00000100:00100000:1.0:1789760761.420162:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771607463600 time 771685935254 state 0x2 ref 0 00000001:00080000:1.0:1789760761.420230:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973566 is committed 00000100:00100000:1.0:1789760761.432541:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760761.432719:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493783552 00000100:00100000:2.0:1789760761.432731:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493783552 00000100:00100000:2.0:1789760761.432805:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771607463600 time 771685935254 state 0x0 ref 1 00000100:00100000:2.0:1789760761.432823:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771686195670 time 771685935254 state 0x1 ref 1 00000100:00100000:2.0:1789760761.432858:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771686195670 time 771773893281 state 0x0 ref 1 00000100:00100000:2.0:1789760761.432873:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493783552:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760761.432937:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25591808, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc807 00010000:00010000:2.0:1789760761.432948:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e06102 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.433007:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:2.0:1789760761.433013:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.433021:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e06102 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.433037:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e06102 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760761.513146:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25591808 -> 25595904 00010000:00010000:1.0:1789760761.513173:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e06102 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.513189:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e06102 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.513201:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e06102 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.513209:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760761.513215:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e06102 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.513226:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e06102 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.513251:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e06102 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.513260:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e06102 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.513397:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493783552:12345-192.168.203.60@tcp:4:dd.0 Request processed in 80510us (80853us total) trans 4294973567 rc 0/0 00000100:00100000:1.0:1789760761.513410:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6278 00000100:00100000:1.0:1789760761.513423:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771686195670 time 771773893281 state 0x2 ref 0 00000001:00080000:1.0:1789760761.513477:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973567 is committed 00000100:00100000:2.0:1789760761.520203:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760761.520492:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493783680 00000100:00100000:0.0:1789760761.520506:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493783680 00000100:00100000:0.0:1789760761.520583:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771686195670 time 771773893281 state 0x0 ref 1 00000100:00100000:0.0:1789760761.520603:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771774153697 time 771773893281 state 0x1 ref 1 00000100:00100000:0.0:1789760761.520640:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771774153697 time 771861675427 state 0x0 ref 1 00000100:00100000:0.0:1789760761.520656:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493783680:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760761.520725:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25595904, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc807 00010000:00010000:0.0:1789760761.520736:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06109 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.520756:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760761.520762:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760761.520770:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06109 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.520787:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06109 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760761.549786:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25595904 -> 25600000 00010000:00010000:0.0:1789760761.549828:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06109 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.549845:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06109 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.549856:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06109 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.549866:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760761.549872:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06109 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.549882:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06109 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.549907:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06109 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.549917:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06109 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760761.550068:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493783680:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29408us (29874us total) trans 4294973568 rc 0/0 00000100:00100000:0.0:1789760761.550081:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6279 00000100:00100000:0.0:1789760761.550094:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771774153697 time 771861675427 state 0x2 ref 0 00000001:00080000:0.0:1789760761.550143:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973568 is committed 00000100:00100000:0.0:1789760761.553853:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760761.554007:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493783808 00000100:00100000:1.0:1789760761.554020:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493783808 00000100:00100000:1.0:1789760761.554093:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771774153697 time 771861675427 state 0x0 ref 1 00000100:00100000:1.0:1789760761.554116:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771861935843 time 771861675427 state 0x1 ref 1 00000100:00100000:1.0:1789760761.554153:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771861935843 time 771895187981 state 0x0 ref 1 00000100:00100000:1.0:1789760761.554167:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493783808:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760761.554231:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25600000, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc807 00010000:00010000:1.0:1789760761.554241:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06110 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.554261:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:1.0:1789760761.554266:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.554274:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06110 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.554290:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06110 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760761.584781:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973569 is committed 00002000:00010000:1.0:1789760761.585002:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25600000 -> 25604096 00010000:00010000:1.0:1789760761.585023:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06110 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.585037:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06110 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.585047:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06110 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.585061:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760761.585067:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06110 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.585077:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06110 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.585102:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06110 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.585111:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06110 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.585280:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493783808:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31107us (31433us total) trans 4294973569 rc 0/0 00000100:00100000:1.0:1789760761.585293:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6280 00000100:00100000:1.0:1789760761.585305:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771861935843 time 771895187981 state 0x2 ref 0 00000100:00100000:3.0:1789760761.590788:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760761.591199:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493783936 00000100:00100000:1.0:1789760761.591215:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493783936 00000100:00100000:1.0:1789760761.591294:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771861935843 time 771895187981 state 0x0 ref 1 00000100:00100000:1.0:1789760761.591313:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771895448397 time 771895187981 state 0x1 ref 1 00000100:00100000:1.0:1789760761.591359:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771895448397 time 771932393812 state 0x0 ref 1 00000100:00100000:1.0:1789760761.591389:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493783936:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760761.591457:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25604096, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xc807 00010000:00010000:1.0:1789760761.591468:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06117 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.592496:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=771s v=(0 0) 00010000:00010000:1.0:1789760761.592508:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.592517:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06117 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.592538:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06117 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760761.672489:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973570 is committed 00002000:00010000:1.0:1789760761.672586:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25604096 -> 25608192 00002000:00010000:1.0:1789760761.672592:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760760 -> 1789760761 00002000:00010000:1.0:1789760761.672595:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760760 -> 1789760761 00010000:00010000:1.0:1789760761.672611:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06117 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.672627:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06117 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.672637:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06117 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.672652:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760761.672658:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06117 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.672668:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06117 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.672695:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06117 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.672704:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06117 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.673044:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493783936:12345-192.168.203.60@tcp:4:dd.0 Request processed in 81660us (82258us total) trans 4294973570 rc 0/0 00000100:00100000:1.0:1789760761.673056:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6281 00000100:00100000:1.0:1789760761.673068:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771895448397 time 771932393812 state 0x2 ref 0 00000100:00100000:0.0:1789760761.676612:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760761.676723:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493784064 00000100:00100000:3.0:1789760761.676737:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493784064 00000100:00100000:3.0:1789760761.676884:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771895448397 time 771932393812 state 0x0 ref 1 00000100:00100000:3.0:1789760761.676906:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771932654228 time 771932393812 state 0x1 ref 1 00000100:00100000:3.0:1789760761.677016:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771932654228 time 772018049770 state 0x0 ref 1 00000100:00100000:3.0:1789760761.677033:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493784064:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760761.677101:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25608192, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc807 00010000:00010000:3.0:1789760761.677113:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0611e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.677134:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760761.677140:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.677147:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0611e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.677164:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0611e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760761.709785:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25608192 -> 25612288 00010000:00010000:3.0:1789760761.709955:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0611e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.710049:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0611e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.710060:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0611e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.710069:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760761.710075:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0611e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.710086:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0611e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.710114:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0611e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.710122:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0611e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.710277:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493784064:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33241us (33672us total) trans 4294973571 rc 0/0 00000100:00100000:3.0:1789760761.710290:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6282 00000100:00100000:3.0:1789760761.710302:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771932654228 time 772018049770 state 0x2 ref 0 00000001:00080000:1.0:1789760761.710415:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973571 is committed 00000100:00100000:1.0:1789760761.714141:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760761.714226:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493784192 00000100:00100000:0.0:1789760761.714240:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493784192 00000100:00100000:0.0:1789760761.714314:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 771932654228 time 772018049770 state 0x0 ref 1 00000100:00100000:0.0:1789760761.714344:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772018310186 time 772018049770 state 0x1 ref 1 00000100:00100000:0.0:1789760761.714374:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772018310186 time 772055410238 state 0x0 ref 1 00000100:00100000:0.0:1789760761.714443:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493784192:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760761.714504:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25612288, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc807 00010000:00010000:0.0:1789760761.714514:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06125 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.714532:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:0.0:1789760761.714715:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760761.714724:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06125 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.714742:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06125 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760761.776728:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973572 is committed 00002000:00010000:0.0:1789760761.777049:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25612288 -> 25616384 00010000:00010000:0.0:1789760761.777070:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06125 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.777086:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06125 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.777096:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06125 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.777105:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760761.777111:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06125 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.777121:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06125 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.777146:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06125 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.777155:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06125 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760761.777312:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493784192:12345-192.168.203.60@tcp:4:dd.0 Request processed in 62865us (63177us total) trans 4294973572 rc 0/0 00000100:00100000:0.0:1789760761.777334:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6283 00000100:00100000:0.0:1789760761.777346:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772018310186 time 772055410238 state 0x2 ref 0 00000100:00100000:1.0:1789760761.780881:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760761.781164:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493784320 00000100:00100000:2.0:1789760761.781178:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493784320 00000100:00100000:2.0:1789760761.781256:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772018310186 time 772055410238 state 0x0 ref 1 00000100:00100000:2.0:1789760761.781275:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772055670654 time 772055410238 state 0x1 ref 1 00000100:00100000:2.0:1789760761.781356:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772055670654 time 772122391229 state 0x0 ref 1 00000100:00100000:2.0:1789760761.781372:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493784320:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760761.781511:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25616384, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc807 00010000:00010000:2.0:1789760761.781522:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0612c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.781542:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:2.0:1789760761.781548:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.781555:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0612c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.781572:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0612c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760761.816708:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973573 is committed 00002000:00010000:0.0:1789760761.816765:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25616384 -> 25620480 00010000:00010000:0.0:1789760761.816786:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0612c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.816800:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0612c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.816878:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0612c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.816892:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760761.816898:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0612c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.816908:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0612c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.816933:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0612c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.816941:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0612c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760761.817132:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493784320:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35755us (36259us total) trans 4294973573 rc 0/0 00000100:00100000:0.0:1789760761.817145:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6284 00000100:00100000:0.0:1789760761.817158:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772055670654 time 772122391229 state 0x2 ref 0 00000100:00100000:0.0:1789760761.819912:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760761.820110:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493784448 00000100:00100000:2.0:1789760761.820122:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493784448 00000100:00100000:2.0:1789760761.820193:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772055670654 time 772122391229 state 0x0 ref 1 00000100:00100000:2.0:1789760761.820212:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772122651645 time 772122391229 state 0x1 ref 1 00000100:00100000:2.0:1789760761.820245:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772122651645 time 772161280237 state 0x0 ref 1 00000100:00100000:2.0:1789760761.820259:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a37480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493784448:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760761.820324:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25620480, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc807 00010000:00010000:2.0:1789760761.820345:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06133 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.820373:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:2.0:1789760761.820378:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.820385:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06133 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.820401:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06133 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760761.864133:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973574 is committed 00002000:00010000:2.0:1789760761.864267:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25620480 -> 25624576 00010000:00010000:2.0:1789760761.864289:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06133 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.864304:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06133 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.864315:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06133 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.864323:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760761.864339:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06133 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.864349:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06133 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.864372:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06133 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.864390:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06133 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.864524:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a37480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493784448:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44260us (44619us total) trans 4294973574 rc 0/0 00000100:00100000:2.0:1789760761.864538:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6285 00000100:00100000:2.0:1789760761.864550:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772122651645 time 772161280237 state 0x2 ref 0 00000100:00100000:2.0:1789760761.868249:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760761.868359:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493784576 00000100:00100000:3.0:1789760761.868371:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493784576 00000100:00100000:3.0:1789760761.868546:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772122651645 time 772161280237 state 0x0 ref 1 00000100:00100000:3.0:1789760761.868564:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772161540653 time 772161280237 state 0x1 ref 1 00000100:00100000:3.0:1789760761.868688:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772161540653 time 772209722700 state 0x0 ref 1 00000100:00100000:3.0:1789760761.868703:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493784576:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760761.868763:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25624576, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc807 00010000:00010000:3.0:1789760761.868774:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0613a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.868794:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760761.868798:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.868861:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0613a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.868880:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0613a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760761.916645:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973575 is committed 00002000:00010000:1.0:1789760761.916792:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25624576 -> 25628672 00010000:00010000:1.0:1789760761.916821:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0613a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.916844:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0613a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.916854:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0613a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.916862:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760761.916869:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0613a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.916880:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0613a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.916906:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0613a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.916915:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0613a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.917081:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493784576:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48373us (48837us total) trans 4294973575 rc 0/0 00000100:00100000:1.0:1789760761.917094:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6286 00000100:00100000:1.0:1789760761.917107:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772161540653 time 772209722700 state 0x2 ref 0 00000100:00100000:0.0:1789760761.920728:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760761.920904:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493784704 00000100:00100000:2.0:1789760761.920918:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493784704 00000100:00100000:2.0:1789760761.921149:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772161540653 time 772209722700 state 0x0 ref 1 00000100:00100000:2.0:1789760761.921168:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772209983116 time 772209722700 state 0x1 ref 1 00000100:00100000:2.0:1789760761.921204:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772209983116 time 772262239482 state 0x0 ref 1 00000100:00100000:2.0:1789760761.921218:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a34e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493784704:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760761.921285:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25628672, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc807 00010000:00010000:2.0:1789760761.921295:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06141 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.921316:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:2.0:1789760761.921321:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.921337:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06141 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.921355:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06141 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760761.967669:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973576 is committed 00002000:00010000:2.0:1789760761.967956:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25628672 -> 25632768 00010000:00010000:2.0:1789760761.967997:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06141 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.968015:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06141 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.968025:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06141 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.968033:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760761.968039:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06141 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.968049:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06141 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.968073:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06141 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.968082:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06141 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760761.968217:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a34e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493784704:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46995us (47498us total) trans 4294973576 rc 0/0 00000100:00100000:2.0:1789760761.968229:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6287 00000100:00100000:2.0:1789760761.968242:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772209983116 time 772262239482 state 0x2 ref 0 00000100:00100000:0.0:1789760761.972045:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760761.972282:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493784832 00000100:00100000:2.0:1789760761.972296:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493784832 00000100:00100000:2.0:1789760761.975713:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772209983116 time 772262239482 state 0x0 ref 1 00000100:00100000:2.0:1789760761.975745:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772262499898 time 772262239482 state 0x1 ref 1 00000100:00100000:2.0:1789760761.976006:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772262499898 time 772317039409 state 0x0 ref 1 00000100:00100000:2.0:1789760761.976039:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493784832:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760761.976112:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25632768, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc807 00010000:00010000:2.0:1789760761.976123:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06148 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.976144:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:2.0:1789760761.976149:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760761.976157:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06148 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760761.976173:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06148 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760762.017284:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973577 is committed 00002000:00010000:3.0:1789760762.018626:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25632768 -> 25636864 00010000:00010000:3.0:1789760762.018652:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06148 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.018667:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06148 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.018684:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06148 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.018692:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760762.018698:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06148 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.018708:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06148 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.018733:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06148 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.018741:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06148 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.018894:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493784832:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42867us (46856us total) trans 4294973577 rc 0/0 00000100:00100000:3.0:1789760762.018906:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6288 00000100:00100000:3.0:1789760762.018918:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772262499898 time 772317039409 state 0x2 ref 0 00000100:00100000:0.0:1789760762.022586:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760762.022698:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493784960 00000100:00100000:1.0:1789760762.022711:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493784960 00000100:00100000:1.0:1789760762.023109:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772262499898 time 772317039409 state 0x0 ref 1 00000100:00100000:1.0:1789760762.023132:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772317299825 time 772317039409 state 0x1 ref 1 00000100:00100000:1.0:1789760762.023171:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772317299825 time 772364206492 state 0x0 ref 1 00000100:00100000:1.0:1789760762.023186:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493784960:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760762.023249:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25636864, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc807 00010000:00010000:1.0:1789760762.023260:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0614f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.023279:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760762.023285:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.023292:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0614f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.023309:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0614f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760762.085611:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973578 is committed 00002000:00010000:3.0:1789760762.085744:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25636864 -> 25640960 00010000:00010000:3.0:1789760762.085765:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0614f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.085788:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0614f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.085799:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0614f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.085818:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760762.085824:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0614f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.085835:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0614f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.085869:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0614f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.085878:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0614f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.086047:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493784960:12345-192.168.203.60@tcp:4:dd.0 Request processed in 62855us (63467us total) trans 4294973578 rc 0/0 00000100:00100000:3.0:1789760762.086060:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6289 00000100:00100000:3.0:1789760762.086074:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772317299825 time 772364206492 state 0x2 ref 0 00000100:00100000:2.0:1789760762.089268:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760762.089365:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493785088 00000100:00100000:3.0:1789760762.089491:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493785088 00000100:00100000:3.0:1789760762.089568:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772317299825 time 772364206492 state 0x0 ref 1 00000100:00100000:3.0:1789760762.089587:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772364466908 time 772364206492 state 0x1 ref 1 00000100:00100000:3.0:1789760762.089627:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772364466908 time 772430661488 state 0x0 ref 1 00000100:00100000:3.0:1789760762.089641:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493785088:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760762.089706:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25640960, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc807 00010000:00010000:3.0:1789760762.089717:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06156 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.089739:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760762.089744:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.089752:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06156 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.089770:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06156 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.134176:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25640960 -> 25645056 00010000:00010000:3.0:1789760762.134201:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06156 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.134215:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06156 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.134229:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06156 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.134237:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760762.134242:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06156 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.134251:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06156 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.134275:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06156 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.134283:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06156 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.134493:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493785088:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44847us (45231us total) trans 4294973579 rc 0/0 00000100:00100000:3.0:1789760762.134506:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6290 00000100:00100000:3.0:1789760762.134518:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772364466908 time 772430661488 state 0x2 ref 0 00000001:00080000:0.0:1789760762.134786:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973579 is committed 00000100:00100000:0.0:1789760762.138068:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760762.138194:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493785216 00000100:00100000:2.0:1789760762.138209:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493785216 00000100:00100000:2.0:1789760762.138293:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772364466908 time 772430661488 state 0x0 ref 1 00000100:00100000:2.0:1789760762.138315:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772430921904 time 772430661488 state 0x1 ref 1 00000100:00100000:2.0:1789760762.138361:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772430921904 time 772479395577 state 0x0 ref 1 00000100:00100000:2.0:1789760762.138386:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493785216:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760762.138584:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25645056, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc807 00010000:00010000:2.0:1789760762.138597:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0615d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.138618:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:2.0:1789760762.138623:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.138631:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0615d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.138648:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0615d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.172013:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973580 is committed 00002000:00010000:1.0:1789760762.172164:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25645056 -> 25649152 00010000:00010000:1.0:1789760762.172183:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0615d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.172198:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0615d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.172215:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0615d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.172223:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760762.172228:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0615d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.172239:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0615d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.172262:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0615d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.172271:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0615d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.172508:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493785216:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34127us (34445us total) trans 4294973580 rc 0/0 00000100:00100000:1.0:1789760762.172521:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6291 00000100:00100000:1.0:1789760762.172533:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772430921904 time 772479395577 state 0x2 ref 0 00000100:00100000:2.0:1789760762.176886:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760762.177057:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493785344 00000100:00100000:3.0:1789760762.177070:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493785344 00000100:00100000:3.0:1789760762.177148:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772430921904 time 772479395577 state 0x0 ref 1 00000100:00100000:3.0:1789760762.177180:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772479655993 time 772479395577 state 0x1 ref 1 00000100:00100000:3.0:1789760762.177216:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772479655993 time 772518251361 state 0x0 ref 1 00000100:00100000:3.0:1789760762.177231:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493785344:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760762.177301:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25649152, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc807 00010000:00010000:3.0:1789760762.177313:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06164 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.177345:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760762.177354:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.177362:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06164 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.177541:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06164 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760762.217748:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973581 is committed 00002000:00010000:3.0:1789760762.217947:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25649152 -> 25653248 00010000:00010000:3.0:1789760762.218054:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06164 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.218069:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06164 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.218079:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06164 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.218092:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760762.218097:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06164 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.218108:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06164 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.218134:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06164 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.218143:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06164 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.218394:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493785344:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41158us (41515us total) trans 4294973581 rc 0/0 00000100:00100000:0.0:1789760762.218408:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6292 00000100:00100000:0.0:1789760762.218420:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772479655993 time 772518251361 state 0x2 ref 0 00000100:00100000:2.0:1789760762.222038:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760762.222179:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493785472 00000100:00100000:3.0:1789760762.222191:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493785472 00000100:00100000:0.0:1789760762.222359:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772479655993 time 772518251361 state 0x0 ref 1 00000100:00100000:0.0:1789760762.222494:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772518511777 time 772518251361 state 0x1 ref 1 00000100:00100000:0.0:1789760762.222582:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772518511777 time 772563616590 state 0x0 ref 1 00000100:00100000:0.0:1789760762.222597:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493785472:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760762.222662:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25653248, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc807 00010000:00010000:0.0:1789760762.222673:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0616b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.222692:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:0.0:1789760762.222698:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.222705:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0616b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.222723:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0616b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760762.259270:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973582 is committed 00002000:00010000:1.0:1789760762.259451:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25653248 -> 25657344 00010000:00010000:1.0:1789760762.259471:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0616b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.259494:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0616b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.259505:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0616b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.259512:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760762.259518:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0616b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.259528:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0616b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.260170:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0616b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.260182:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0616b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.260347:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493785472:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37746us (38388us total) trans 4294973582 rc 0/0 00000100:00100000:1.0:1789760762.260359:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6293 00000100:00100000:1.0:1789760762.260371:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772518511777 time 772563616590 state 0x2 ref 0 00000100:00100000:0.0:1789760762.263111:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760762.263213:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493785600 00000100:00100000:1.0:1789760762.263225:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493785600 00000100:00100000:1.0:1789760762.263300:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772518511777 time 772563616590 state 0x0 ref 1 00000100:00100000:1.0:1789760762.263319:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772563877006 time 772563616590 state 0x1 ref 1 00000100:00100000:1.0:1789760762.263361:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772563877006 time 772604395719 state 0x0 ref 1 00000100:00100000:1.0:1789760762.263384:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc14e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493785600:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760762.263446:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25657344, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc807 00010000:00010000:1.0:1789760762.263457:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06172 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.263476:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760762.263480:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.263488:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06172 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.263504:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06172 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.290757:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973583 is committed 00002000:00010000:1.0:1789760762.290836:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25657344 -> 25661440 00010000:00010000:1.0:1789760762.290856:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06172 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.290871:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06172 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.290883:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06172 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.290888:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760762.290893:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06172 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.290906:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06172 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.290927:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06172 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.290932:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06172 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.291101:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc14e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493785600:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27722us (27996us total) trans 4294973583 rc 0/0 00000100:00100000:1.0:1789760762.291113:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6294 00000100:00100000:1.0:1789760762.291124:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772563877006 time 772604395719 state 0x2 ref 0 00000100:00100000:0.0:1789760762.294319:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760762.295483:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493785728 00000100:00100000:1.0:1789760762.295498:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493785728 00000100:00100000:1.0:1789760762.295651:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772563877006 time 772604395719 state 0x0 ref 1 00000100:00100000:1.0:1789760762.295671:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772604656135 time 772604395719 state 0x1 ref 1 00000100:00100000:1.0:1789760762.295751:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772604656135 time 772636785185 state 0x0 ref 1 00000100:00100000:1.0:1789760762.295767:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493785728:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760762.295845:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25661440, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc807 00010000:00010000:1.0:1789760762.295856:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06179 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.295877:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760762.295882:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.295890:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06179 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.295917:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06179 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.341594:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973584 is committed 00002000:00010000:2.0:1789760762.341782:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25661440 -> 25665536 00010000:00010000:2.0:1789760762.341803:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06179 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.341818:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06179 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.341828:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06179 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.341835:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760762.341841:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06179 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.341851:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06179 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.341876:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06179 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.342129:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06179 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.342297:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493785728:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46526us (47985us total) trans 4294973584 rc 0/0 00000100:00100000:2.0:1789760762.342310:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6295 00000100:00100000:2.0:1789760762.342334:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772604656135 time 772636785185 state 0x2 ref 0 00000100:00100000:2.0:1789760762.346140:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760762.346240:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493785856 00000100:00100000:0.0:1789760762.346253:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493785856 00000100:00100000:0.0:1789760762.346343:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772604656135 time 772636785185 state 0x0 ref 1 00000100:00100000:0.0:1789760762.346362:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772637045601 time 772636785185 state 0x1 ref 1 00000100:00100000:0.0:1789760762.346491:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772637045601 time 772687525146 state 0x0 ref 1 00000100:00100000:0.0:1789760762.346506:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb04050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493785856:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760762.346587:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25665536, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc807 00010000:00010000:0.0:1789760762.346598:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06180 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.346619:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:0.0:1789760762.346624:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.346632:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06180 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.346648:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06180 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760762.398704:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25665536 -> 25669632 00000001:00080000:3.0:1789760762.398730:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973585 is committed 00010000:00010000:2.0:1789760762.398734:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06180 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.398748:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06180 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.398771:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06180 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.398779:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760762.398783:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06180 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.398791:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06180 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.398813:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06180 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.398818:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06180 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.399288:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb04050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493785856:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52772us (53150us total) trans 4294973585 rc 0/0 00000100:00100000:2.0:1789760762.399302:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6296 00000100:00100000:2.0:1789760762.399315:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772637045601 time 772687525146 state 0x2 ref 0 00000100:00100000:0.0:1789760762.404037:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760762.404366:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493785984 00000100:00100000:1.0:1789760762.404383:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493785984 00000100:00100000:1.0:1789760762.404468:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772637045601 time 772687525146 state 0x0 ref 1 00000100:00100000:1.0:1789760762.404487:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772687785562 time 772687525146 state 0x1 ref 1 00000100:00100000:1.0:1789760762.404526:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772687785562 time 772745560293 state 0x0 ref 1 00000100:00100000:1.0:1789760762.404542:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493785984:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760762.404609:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25669632, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc807 00010000:00010000:1.0:1789760762.404622:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06187 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.404641:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760762.404646:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.404654:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06187 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.404671:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06187 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.426294:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760762.450002:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973586 is committed 00002000:00010000:1.0:1789760762.450052:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25669632 -> 25673728 00010000:00010000:1.0:1789760762.450073:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06187 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.450088:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06187 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.450097:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06187 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.450105:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760762.450110:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06187 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.450120:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06187 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.450151:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06187 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.450160:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06187 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.450283:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493785984:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45738us (46251us total) trans 4294973586 rc 0/0 00000100:00100000:1.0:1789760762.450295:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6297 00000100:00100000:1.0:1789760762.450308:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772687785562 time 772745560293 state 0x2 ref 0 00000100:00100000:0.0:1789760762.453626:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760762.453722:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493786112 00000100:00100000:1.0:1789760762.453735:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493786112 00000100:00100000:1.0:1789760762.453807:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772687785562 time 772745560293 state 0x0 ref 1 00000100:00100000:1.0:1789760762.453826:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772745820709 time 772745560293 state 0x1 ref 1 00000100:00100000:1.0:1789760762.453864:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772745820709 time 772794898929 state 0x0 ref 1 00000100:00100000:1.0:1789760762.453879:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc14e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493786112:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760762.453947:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25673728, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc807 00010000:00010000:1.0:1789760762.453998:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0618e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.454019:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760762.454026:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.454033:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0618e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.454051:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0618e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760762.502492:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973587 is committed 00002000:00010000:1.0:1789760762.502535:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25673728 -> 25677824 00010000:00010000:1.0:1789760762.502557:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0618e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.502572:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0618e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.502583:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0618e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.502592:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760762.502598:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0618e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.502610:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0618e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.502635:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0618e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.502644:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0618e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.502775:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc14e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493786112:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48893us (49204us total) trans 4294973587 rc 0/0 00000100:00100000:1.0:1789760762.502788:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6298 00000100:00100000:1.0:1789760762.503480:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772745820709 time 772794898929 state 0x2 ref 0 00000100:00100000:3.0:1789760762.506263:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760762.506483:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493786240 00000100:00100000:2.0:1789760762.506496:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493786240 00000100:00100000:2.0:1789760762.506574:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772745820709 time 772794898929 state 0x0 ref 1 00000100:00100000:2.0:1789760762.506593:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772795159345 time 772794898929 state 0x1 ref 1 00000100:00100000:2.0:1789760762.506631:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772795159345 time 772847665083 state 0x0 ref 1 00000100:00100000:2.0:1789760762.506645:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493786240:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760762.506710:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25677824, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc807 00010000:00010000:2.0:1789760762.506720:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06195 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.506739:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:2.0:1789760762.506744:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.506752:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06195 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.506769:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06195 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.581133:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973588 is committed 00002000:00010000:1.0:1789760762.581238:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25677824 -> 25681920 00010000:00010000:1.0:1789760762.581253:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06195 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.581266:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06195 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.581275:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06195 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.581282:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760762.581287:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06195 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.581296:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06195 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.581315:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06195 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.581337:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06195 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.581551:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493786240:12345-192.168.203.60@tcp:4:dd.0 Request processed in 74904us (75297us total) trans 4294973588 rc 0/0 00000100:00100000:1.0:1789760762.581563:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6299 00000100:00100000:1.0:1789760762.581573:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772795159345 time 772847665083 state 0x2 ref 0 00000100:00100000:2.0:1789760762.588342:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760762.588445:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493786368 00000100:00100000:0.0:1789760762.588454:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493786368 00000100:00100000:0.0:1789760762.588506:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772795159345 time 772847665083 state 0x0 ref 1 00000100:00100000:0.0:1789760762.588520:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772847925499 time 772847665083 state 0x1 ref 1 00000100:00100000:0.0:1789760762.588547:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772847925499 time 772929583743 state 0x0 ref 1 00000100:00100000:0.0:1789760762.588559:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493786368:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760762.588613:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25681920, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xc807 00010000:00010000:0.0:1789760762.588622:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0619c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.588637:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=772s v=(0 0) 00010000:00010000:0.0:1789760762.588642:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.588648:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0619c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.588662:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0619c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760762.664234:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973589 is committed 00002000:00010000:2.0:1789760762.664547:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25681920 -> 25686016 00002000:00010000:2.0:1789760762.664552:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760761 -> 1789760762 00002000:00010000:2.0:1789760762.664555:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760761 -> 1789760762 00010000:00010000:2.0:1789760762.664573:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0619c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.664588:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0619c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.664664:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0619c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.664673:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760762.664682:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0619c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.664693:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0619c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.664720:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0619c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.664728:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0619c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.665029:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493786368:12345-192.168.203.60@tcp:4:dd.0 Request processed in 76462us (76697us total) trans 4294973589 rc 0/0 00000100:00100000:2.0:1789760762.665042:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6300 00000100:00100000:2.0:1789760762.665055:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772847925499 time 772929583743 state 0x2 ref 0 00000100:00100000:0.0:1789760762.678215:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760762.678568:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493786496 00000100:00100000:1.0:1789760762.678581:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493786496 00000100:00100000:1.0:1789760762.678660:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772847925499 time 772929583743 state 0x0 ref 1 00000100:00100000:1.0:1789760762.678850:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772929844159 time 772929583743 state 0x1 ref 1 00000100:00100000:1.0:1789760762.678932:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772929844159 time 773019965306 state 0x0 ref 1 00000100:00100000:1.0:1789760762.678949:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493786496:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760762.679066:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25686016, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc807 00010000:00010000:1.0:1789760762.679078:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061a3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.679100:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760762.679105:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.679112:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.679130:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.730914:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973590 is committed 00002000:00010000:1.0:1789760762.731229:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25686016 -> 25690112 00010000:00010000:1.0:1789760762.731249:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.731265:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061a3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.731275:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.731284:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760762.731290:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.731301:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061a3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.731339:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061a3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.731348:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061a3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.731665:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493786496:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52713us (53460us total) trans 4294973590 rc 0/0 00000100:00100000:1.0:1789760762.731678:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6301 00000100:00100000:1.0:1789760762.731691:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772929844159 time 773019965306 state 0x2 ref 0 00000100:00100000:0.0:1789760762.735165:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760762.735307:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493786624 00000100:00100000:1.0:1789760762.735335:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493786624 00000100:00100000:1.0:1789760762.735423:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 772929844159 time 773019965306 state 0x0 ref 1 00000100:00100000:1.0:1789760762.735442:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773020225722 time 773019965306 state 0x1 ref 1 00000100:00100000:1.0:1789760762.735479:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773020225722 time 773076513759 state 0x0 ref 1 00000100:00100000:1.0:1789760762.735492:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493786624:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760762.735567:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25690112, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc807 00010000:00010000:1.0:1789760762.735577:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e061aa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.735597:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760762.735602:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.735610:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e061aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.735627:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e061aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760762.766278:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973591 is committed 00002000:00010000:1.0:1789760762.766418:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25690112 -> 25694208 00010000:00010000:1.0:1789760762.766436:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e061aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.766449:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e061aa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.766458:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e061aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.766465:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760762.766471:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e061aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.766480:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e061aa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.766503:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e061aa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.766511:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e061aa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.766630:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493786624:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31130us (31671us total) trans 4294973591 rc 0/0 00000100:00100000:1.0:1789760762.766647:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6302 00000100:00100000:1.0:1789760762.766661:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773020225722 time 773076513759 state 0x2 ref 0 00000100:00100000:2.0:1789760762.770316:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760762.770448:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493786752 00000100:00100000:1.0:1789760762.770461:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493786752 00000100:00100000:1.0:1789760762.770538:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773020225722 time 773076513759 state 0x0 ref 1 00000100:00100000:1.0:1789760762.770558:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773076774175 time 773076513759 state 0x1 ref 1 00000100:00100000:1.0:1789760762.770600:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773076774175 time 773111635124 state 0x0 ref 1 00000100:00100000:1.0:1789760762.770615:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493786752:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760762.770693:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25694208, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc807 00010000:00010000:1.0:1789760762.770706:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061b1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.770740:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760762.770745:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.770754:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.770843:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760762.815731:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973592 is committed 00002000:00010000:3.0:1789760762.816005:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25694208 -> 25698304 00010000:00010000:3.0:1789760762.816026:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.816043:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061b1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.816054:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.816061:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760762.816068:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.816078:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061b1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.816101:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061b1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.816110:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061b1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.816272:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493786752:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45653us (45963us total) trans 4294973592 rc 0/0 00000100:00100000:3.0:1789760762.816285:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6303 00000100:00100000:3.0:1789760762.816297:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773076774175 time 773111635124 state 0x2 ref 0 00000100:00100000:0.0:1789760762.819385:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760762.819561:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493786880 00000100:00100000:2.0:1789760762.819574:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493786880 00000100:00100000:2.0:1789760762.819647:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773076774175 time 773111635124 state 0x0 ref 1 00000100:00100000:2.0:1789760762.819665:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773111895540 time 773111635124 state 0x1 ref 1 00000100:00100000:2.0:1789760762.819701:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773111895540 time 773160736164 state 0x0 ref 1 00000100:00100000:2.0:1789760762.819714:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493786880:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760762.820044:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25698304, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc807 00010000:00010000:2.0:1789760762.820055:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e061b8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.820073:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760762.820078:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.820085:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e061b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.820119:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e061b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760762.853938:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973593 is committed 00002000:00010000:3.0:1789760762.854229:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25698304 -> 25702400 00010000:00010000:3.0:1789760762.854250:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e061b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.854273:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e061b8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.854284:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e061b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.854293:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760762.854299:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e061b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.854310:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e061b8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.854346:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e061b8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.854354:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e061b8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.854484:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493786880:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34763us (35112us total) trans 4294973593 rc 0/0 00000100:00100000:3.0:1789760762.854496:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6304 00000100:00100000:3.0:1789760762.854510:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773111895540 time 773160736164 state 0x2 ref 0 00000100:00100000:2.0:1789760762.858432:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760762.859178:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493787008 00000100:00100000:0.0:1789760762.859193:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493787008 00000100:00100000:1.0:1789760762.859529:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773111895540 time 773160736164 state 0x0 ref 1 00000100:00100000:1.0:1789760762.859551:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773160996580 time 773160736164 state 0x1 ref 1 00000100:00100000:1.0:1789760762.859616:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773160996580 time 773200650350 state 0x0 ref 1 00000100:00100000:1.0:1789760762.859630:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493787008:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760762.859697:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25702400, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc807 00010000:00010000:1.0:1789760762.859710:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e061bf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.859730:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760762.859736:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.859744:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e061bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.859760:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e061bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760762.896798:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973594 is committed 00002000:00010000:3.0:1789760762.896934:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25702400 -> 25706496 00010000:00010000:3.0:1789760762.896954:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e061bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.896992:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e061bf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.897002:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e061bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.897010:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760762.897016:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e061bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.897027:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e061bf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.897053:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e061bf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.897062:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e061bf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.897190:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493787008:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37554us (38766us total) trans 4294973594 rc 0/0 00000100:00100000:3.0:1789760762.897202:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6305 00000100:00100000:3.0:1789760762.897215:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773160996580 time 773200650350 state 0x2 ref 0 00000100:00100000:2.0:1789760762.900275:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760762.900544:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493787136 00000100:00100000:3.0:1789760762.900559:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493787136 00000100:00100000:3.0:1789760762.900642:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773160996580 time 773200650350 state 0x0 ref 1 00000100:00100000:3.0:1789760762.900663:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773200910766 time 773200650350 state 0x1 ref 1 00000100:00100000:3.0:1789760762.900704:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773200910766 time 773241739182 state 0x0 ref 1 00000100:00100000:3.0:1789760762.900719:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493787136:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760762.900791:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25706496, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc807 00010000:00010000:3.0:1789760762.900811:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e061c6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.900832:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760762.900837:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.900844:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e061c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.900860:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e061c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760762.950805:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973595 is committed 00002000:00010000:3.0:1789760762.952494:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25706496 -> 25710592 00010000:00010000:3.0:1789760762.952520:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e061c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.952535:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e061c6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.952545:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e061c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.952553:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760762.952559:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e061c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.952570:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e061c6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.952593:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e061c6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.952601:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e061c6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.952749:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493787136:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52021us (52479us total) trans 4294973595 rc 0/0 00000100:00100000:3.0:1789760762.952760:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6306 00000100:00100000:3.0:1789760762.952773:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773200910766 time 773241739182 state 0x2 ref 0 00000100:00100000:0.0:1789760762.970657:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760762.970878:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493787264 00000100:00100000:3.0:1789760762.970893:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493787264 00000100:00100000:3.0:1789760762.971022:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773200910766 time 773241739182 state 0x0 ref 1 00000100:00100000:3.0:1789760762.971042:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773241999598 time 773241739182 state 0x1 ref 1 00000100:00100000:3.0:1789760762.971082:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773241999598 time 773312118146 state 0x0 ref 1 00000100:00100000:3.0:1789760762.971096:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493787264:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760762.971164:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25710592, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc807 00010000:00010000:3.0:1789760762.971175:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e061cd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.971194:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760762.971199:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.971207:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e061cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.971225:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e061cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.994592:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760762.994642:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760762.994652:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493787392 00000100:00100000:3.0:1789760762.994666:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493787392 00000100:00100000:2.0:1789760762.994674:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760762.994707:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493787520 00000100:00100000:2.0:1789760762.994712:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493787520 00000100:00100000:3.0:1789760762.994732:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 407 00000100:00100000:3.0:1789760762.994746:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d4b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2370:x1876699493787392:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:2.0:1789760762.994762:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493787648 00000100:00100000:2.0:1789760762.994767:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493787648 00000100:00100000:2.0:1789760762.994798:0:9460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 502 00000100:00100000:2.0:1789760762.994807:0:9460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2369:x1876699493787520:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:3.0:1789760762.994841:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d4b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2370:x1876699493787392:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 97us (264us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760762.994850:0:9460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2369:x1876699493787520:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 44us (206us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760762.994852:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 407 00000100:00100000:2.0:1789760762.994861:0:9460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 502 00000100:00100000:2.0:1789760762.994884:0:9460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 503 00000100:00100000:2.0:1789760762.994891:0:9460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2370:x1876699493787648:12345-192.168.203.60@tcp:400:kworker.0 00000100:00080000:3.0:1789760762.994915:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760762.994922:0:9460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2370:x1876699493787648:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 32us (247us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760762.994931:0:9460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 503 00000001:00080000:0.0:1789760763.026332:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973596 is committed 00002000:00010000:1.0:1789760763.026505:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25710592 -> 25714688 00010000:00010000:1.0:1789760763.026525:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e061cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.026541:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e061cd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.026550:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e061cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.026564:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760763.026569:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e061cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.026580:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e061cd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.026605:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e061cd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.026614:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e061cd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.026743:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493787264:12345-192.168.203.60@tcp:4:dd.0 Request processed in 55641us (56095us total) trans 4294973596 rc 0/0 00000100:00100000:1.0:1789760763.026756:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6307 00000100:00100000:1.0:1789760763.026768:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773241999598 time 773312118146 state 0x2 ref 0 00000100:00100000:2.0:1789760763.034663:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760763.034772:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493787776 00000100:00100000:3.0:1789760763.034785:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493787776 00000100:00100000:3.0:1789760763.034886:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773241999598 time 773312118146 state 0x0 ref 1 00000100:00100000:3.0:1789760763.034906:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773312378562 time 773312118146 state 0x1 ref 1 00000100:00100000:3.0:1789760763.034940:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773312378562 time 773375975091 state 0x0 ref 1 00000100:00100000:3.0:1789760763.034955:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493787776:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760763.035328:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25714688, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc807 00010000:00010000:2.0:1789760763.035341:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e061d4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.035361:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760763.035366:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760763.035374:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e061d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.035390:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e061d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760763.039155:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760763.077353:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973597 is committed 00002000:00010000:2.0:1789760763.077848:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25714688 -> 25718784 00010000:00010000:2.0:1789760763.077868:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e061d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.077915:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e061d4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.077925:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e061d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.077931:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760763.077935:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e061d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.077942:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e061d4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.077992:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e061d4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.078002:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e061d4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760763.078158:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493787776:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43198us (43600us total) trans 4294973597 rc 0/0 00000100:00100000:2.0:1789760763.078171:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6308 00000100:00100000:2.0:1789760763.078183:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773312378562 time 773375975091 state 0x2 ref 0 00000100:00100000:1.0:1789760763.082245:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760763.082417:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493787904 00000100:00100000:3.0:1789760763.082439:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493787904 00000100:00100000:3.0:1789760763.082512:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773312378562 time 773375975091 state 0x0 ref 1 00000100:00100000:3.0:1789760763.082531:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773376235507 time 773375975091 state 0x1 ref 1 00000100:00100000:3.0:1789760763.082564:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773376235507 time 773423599393 state 0x0 ref 1 00000100:00100000:3.0:1789760763.082578:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493787904:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760763.082644:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25718784, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc807 00010000:00010000:3.0:1789760763.082654:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e061db lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.082673:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760763.082677:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.082684:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e061db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.082701:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e061db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760763.133256:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973598 is committed 00002000:00010000:1.0:1789760763.133496:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25718784 -> 25722880 00010000:00010000:1.0:1789760763.133517:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e061db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.133534:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e061db lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.133545:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e061db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.133553:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760763.133559:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e061db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.133569:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e061db lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.133593:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e061db lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.133602:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e061db lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.133870:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493787904:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51285us (51639us total) trans 4294973598 rc 0/0 00000100:00100000:1.0:1789760763.133883:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6309 00000100:00100000:1.0:1789760763.133896:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773376235507 time 773423599393 state 0x2 ref 0 00000100:00100000:2.0:1789760763.141039:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760763.141187:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493788032 00000100:00100000:1.0:1789760763.141201:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493788032 00000100:00100000:1.0:1789760763.141289:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773376235507 time 773423599393 state 0x0 ref 1 00000100:00100000:1.0:1789760763.141308:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773423859809 time 773423599393 state 0x1 ref 1 00000100:00100000:1.0:1789760763.141350:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773423859809 time 773482384700 state 0x0 ref 1 00000100:00100000:1.0:1789760763.141364:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493788032:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760763.141677:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25722880, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc807 00010000:00010000:1.0:1789760763.141689:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e061e2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.141710:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760763.141888:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.141898:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e061e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.141916:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e061e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760763.241033:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973599 is committed 00002000:00010000:0.0:1789760763.241178:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25722880 -> 25726976 00010000:00010000:0.0:1789760763.241217:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e061e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.241233:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e061e2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.241243:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e061e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.241253:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760763.241260:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e061e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.241272:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e061e2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.241640:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e061e2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.241650:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e061e2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.241808:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493788032:12345-192.168.203.60@tcp:4:dd.0 Request processed in 100443us (100780us total) trans 4294973599 rc 0/0 00000100:00100000:0.0:1789760763.241819:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6310 00000100:00100000:0.0:1789760763.241829:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773423859809 time 773482384700 state 0x2 ref 0 00000100:00100000:2.0:1789760763.250089:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760763.252451:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493788160 00000100:00100000:3.0:1789760763.252471:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493788160 00000100:00100000:3.0:1789760763.252566:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773423859809 time 773482384700 state 0x0 ref 1 00000100:00100000:3.0:1789760763.252587:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773482645116 time 773482384700 state 0x1 ref 1 00000100:00100000:3.0:1789760763.252706:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773482645116 time 773593739112 state 0x0 ref 1 00000100:00100000:3.0:1789760763.252724:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493788160:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760763.252847:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25726976, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc807 00010000:00010000:3.0:1789760763.252861:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e061e9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.252880:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760763.252886:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.252894:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e061e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.252913:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e061e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760763.325453:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973600 is committed 00002000:00010000:0.0:1789760763.325634:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25726976 -> 25731072 00010000:00010000:0.0:1789760763.325655:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e061e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.325671:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e061e9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.325681:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e061e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.325690:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760763.325696:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e061e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.325707:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e061e9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.325731:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e061e9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.325739:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e061e9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.325868:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493788160:12345-192.168.203.60@tcp:4:dd.0 Request processed in 73142us (75786us total) trans 4294973600 rc 0/0 00000100:00100000:0.0:1789760763.325884:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6311 00000100:00100000:0.0:1789760763.325897:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773482645116 time 773593739112 state 0x2 ref 0 00000100:00100000:1.0:1789760763.329819:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760763.330019:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493788288 00000100:00100000:2.0:1789760763.330032:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493788288 00000100:00100000:2.0:1789760763.330111:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773482645116 time 773593739112 state 0x0 ref 1 00000100:00100000:2.0:1789760763.330145:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773593999528 time 773593739112 state 0x1 ref 1 00000100:00100000:2.0:1789760763.330172:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773593999528 time 773671207247 state 0x0 ref 1 00000100:00100000:2.0:1789760763.330188:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493788288:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760763.330259:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25731072, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc807 00010000:00010000:2.0:1789760763.330271:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e061f0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.330290:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760763.330295:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760763.330303:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e061f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.330336:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e061f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760763.390293:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25731072 -> 25735168 00010000:00010000:2.0:1789760763.390333:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e061f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.390350:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e061f0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.390360:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e061f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.390378:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760763.390383:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e061f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.390394:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e061f0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.390421:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e061f0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.390430:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e061f0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760763.390564:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493788288:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60371us (60765us total) trans 4294973601 rc 0/0 00000100:00100000:2.0:1789760763.390576:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6312 00000100:00100000:2.0:1789760763.390589:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773593999528 time 773671207247 state 0x2 ref 0 00000001:00080000:0.0:1789760763.392590:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973601 is committed 00000100:00100000:2.0:1789760763.395106:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760763.395258:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493788416 00000100:00100000:1.0:1789760763.395270:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493788416 00000100:00100000:1.0:1789760763.395360:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773593999528 time 773671207247 state 0x0 ref 1 00000100:00100000:1.0:1789760763.395465:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773671467663 time 773671207247 state 0x1 ref 1 00000100:00100000:1.0:1789760763.395502:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773671467663 time 773736537656 state 0x0 ref 1 00000100:00100000:1.0:1789760763.395517:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493788416:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760763.395592:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25735168, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc807 00010000:00010000:1.0:1789760763.395603:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061f7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.395624:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760763.395630:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.395637:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.395654:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760763.435095:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973602 is committed 00002000:00010000:2.0:1789760763.436105:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25735168 -> 25739264 00010000:00010000:2.0:1789760763.436129:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.436144:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061f7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.436154:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.436162:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760763.436168:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.436184:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061f7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.436209:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061f7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.436218:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e061f7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760763.436655:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493788416:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41133us (41556us total) trans 4294973602 rc 0/0 00000100:00100000:2.0:1789760763.436668:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6313 00000100:00100000:2.0:1789760763.436681:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773671467663 time 773736537656 state 0x2 ref 0 00000100:00100000:2.0:1789760763.440598:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760763.440686:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493788544 00000100:00100000:1.0:1789760763.440699:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493788544 00000100:00100000:1.0:1789760763.440787:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773671467663 time 773736537656 state 0x0 ref 1 00000100:00100000:1.0:1789760763.440932:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773736798072 time 773736537656 state 0x1 ref 1 00000100:00100000:1.0:1789760763.440993:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773736798072 time 773782027453 state 0x0 ref 1 00000100:00100000:1.0:1789760763.441008:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493788544:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760763.441076:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25739264, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc807 00010000:00010000:1.0:1789760763.441087:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e061fe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.441107:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760763.441113:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.441121:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e061fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.441137:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e061fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.452152:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760763.491527:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25739264 -> 25743360 00010000:00010000:0.0:1789760763.491559:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e061fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.491575:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e061fe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.491585:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e061fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.491595:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760763.491601:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e061fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.491612:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e061fe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.491637:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e061fe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.491645:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e061fe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.491782:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493788544:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50760us (51182us total) trans 4294973603 rc 0/0 00000100:00100000:0.0:1789760763.491805:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6314 00000100:00100000:0.0:1789760763.491818:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773736798072 time 773782027453 state 0x2 ref 0 00000001:00080000:0.0:1789760763.491872:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973603 is committed 00000100:00100000:2.0:1789760763.495758:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760763.495882:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493788672 00000100:00100000:0.0:1789760763.495895:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493788672 00000100:00100000:0.0:1789760763.496015:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773736798072 time 773782027453 state 0x0 ref 1 00000100:00100000:0.0:1789760763.496034:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773782287869 time 773782027453 state 0x1 ref 1 00000100:00100000:0.0:1789760763.496065:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773782287869 time 773837100033 state 0x0 ref 1 00000100:00100000:0.0:1789760763.496079:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493788672:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760763.496144:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25743360, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc807 00010000:00010000:0.0:1789760763.496156:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06205 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.496175:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760763.496181:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.496189:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06205 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.496208:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06205 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760763.537012:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973604 is committed 00002000:00010000:3.0:1789760763.537108:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25743360 -> 25747456 00010000:00010000:3.0:1789760763.537128:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06205 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.537149:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06205 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.537158:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06205 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.537165:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760763.537171:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06205 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.537180:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06205 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.537203:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06205 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.537210:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06205 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.537336:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493788672:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41252us (41584us total) trans 4294973604 rc 0/0 00000100:00100000:3.0:1789760763.537348:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6315 00000100:00100000:3.0:1789760763.537359:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773782287869 time 773837100033 state 0x2 ref 0 00000100:00100000:2.0:1789760763.544189:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760763.544257:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493788800 00000100:00100000:3.0:1789760763.544272:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493788800 00000100:00100000:3.0:1789760763.544357:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773782287869 time 773837100033 state 0x0 ref 1 00000100:00100000:3.0:1789760763.544377:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773837360449 time 773837100033 state 0x1 ref 1 00000100:00100000:3.0:1789760763.544492:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773837360449 time 773885445454 state 0x0 ref 1 00000100:00100000:3.0:1789760763.544509:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493788800:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760763.544577:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25747456, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc807 00010000:00010000:3.0:1789760763.544588:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0620c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.544606:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760763.544611:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.544620:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0620c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.544636:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0620c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.618627:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25747456 -> 25751552 00010000:00010000:3.0:1789760763.618654:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0620c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.618670:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0620c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.618700:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0620c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.618709:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760763.618714:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0620c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.618724:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0620c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.618749:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0620c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.618758:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0620c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.618886:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493788800:12345-192.168.203.60@tcp:4:dd.0 Request processed in 74374us (74703us total) trans 4294973605 rc 0/0 00000100:00100000:3.0:1789760763.618899:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6316 00000100:00100000:3.0:1789760763.618911:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773837360449 time 773885445454 state 0x2 ref 0 00000001:00080000:0.0:1789760763.619568:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973605 is committed 00000100:00100000:0.0:1789760763.621410:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760763.621587:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493788928 00000100:00100000:2.0:1789760763.621600:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493788928 00000100:00100000:2.0:1789760763.622198:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773837360449 time 773885445454 state 0x0 ref 1 00000100:00100000:2.0:1789760763.622219:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773885705870 time 773885445454 state 0x1 ref 1 00000100:00100000:2.0:1789760763.622296:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773885705870 time 773963330873 state 0x0 ref 1 00000100:00100000:2.0:1789760763.622322:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493788928:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760763.622456:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25751552, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xc807 00010000:00010000:2.0:1789760763.622468:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06213 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.622489:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760763.622495:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760763.622503:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06213 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.622520:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06213 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760763.678470:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973606 is committed 00002000:00010000:0.0:1789760763.678641:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25751552 -> 25755648 00002000:00010000:0.0:1789760763.678647:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760762 -> 1789760763 00002000:00010000:0.0:1789760763.678650:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760762 -> 1789760763 00010000:00010000:0.0:1789760763.678667:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06213 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.678683:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06213 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.678693:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06213 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.678702:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760763.678708:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06213 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.678718:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06213 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.678744:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06213 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.678758:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06213 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.678922:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493788928:12345-192.168.203.60@tcp:4:dd.0 Request processed in 56608us (57519us total) trans 4294973606 rc 0/0 00000100:00100000:0.0:1789760763.678936:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6317 00000100:00100000:0.0:1789760763.678949:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773885705870 time 773963330873 state 0x2 ref 0 00000100:00100000:0.0:1789760763.687446:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760763.694844:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493789056 00000100:00100000:1.0:1789760763.694868:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493789056 00000100:00100000:0.0:1789760763.695073:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773885705870 time 773963330873 state 0x0 ref 1 00000100:00100000:0.0:1789760763.695092:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773963591289 time 773963330873 state 0x1 ref 1 00000100:00100000:0.0:1789760763.695133:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773963591289 time 774036167524 state 0x0 ref 1 00000100:00100000:0.0:1789760763.695149:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493789056:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760763.695231:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25755648, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc807 00010000:00010000:0.0:1789760763.695243:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0621a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.695262:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:0.0:1789760763.695268:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.695277:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0621a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.695300:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0621a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760763.760650:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973607 is committed 00002000:00010000:1.0:1789760763.760858:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25755648 -> 25759744 00010000:00010000:1.0:1789760763.760877:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0621a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.760892:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0621a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.760902:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0621a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.760910:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760763.760916:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0621a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.760926:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0621a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.760949:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0621a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.760980:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0621a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.761097:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493789056:12345-192.168.203.60@tcp:4:dd.0 Request processed in 65944us (73661us total) trans 4294973607 rc 0/0 00000100:00100000:1.0:1789760763.761109:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6318 00000100:00100000:1.0:1789760763.761120:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773963591289 time 774036167524 state 0x2 ref 0 00000100:00100000:1.0:1789760763.770305:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc48de300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3303:1876699519256960:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760763.770417:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760763.770619:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519256960 00000100:00100000:3.0:1789760763.770630:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519256960 00000100:00100000:3.0:1789760763.770728:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 208 00000100:00100000:3.0:1789760763.770739:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876699519256960:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789760763.770785:0:6690:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526450688 pending 0 free 3832545280 avail 3811573760 00000100:00080000:0.0:1789760763.772236:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760763.775552:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876699519256960:12345-0@lo:13:osp-pre-0-0.0 Request processed in 4801us (5120us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760763.775569:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 208 00000100:00100000:0.0:1789760763.779931:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760763.780154:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493789184 00000100:00100000:3.0:1789760763.780168:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493789184 00000100:00100000:3.0:1789760763.780244:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 773963591289 time 774036167524 state 0x0 ref 1 00000100:00100000:3.0:1789760763.780263:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774036427940 time 774036167524 state 0x1 ref 1 00000100:00100000:3.0:1789760763.780297:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774036427940 time 774121332306 state 0x0 ref 1 00000100:00100000:3.0:1789760763.780321:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493789184:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760763.781261:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25759744, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc807 00010000:00010000:3.0:1789760763.781277:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06221 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.781295:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:3.0:1789760763.781300:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.781306:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06221 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.781332:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06221 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760763.784904:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc48de300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3303:1876699519256960:0@lo:13:osp-pre-0-0.0 00000001:00080000:2.0:1789760763.832716:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973608 is committed 00002000:00010000:3.0:1789760763.833007:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25759744 -> 25763840 00010000:00010000:3.0:1789760763.833027:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06221 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.833042:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06221 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.833052:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06221 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.833060:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760763.833066:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06221 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.833076:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06221 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.833100:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06221 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.833107:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06221 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.833302:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493789184:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52985us (53379us total) trans 4294973608 rc 0/0 00000100:00100000:3.0:1789760763.833323:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6319 00000100:00100000:3.0:1789760763.833335:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774036427940 time 774121332306 state 0x2 ref 0 00000100:00100000:1.0:1789760763.842119:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760763.844570:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493789312 00000100:00100000:3.0:1789760763.844590:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493789312 00000100:00100000:3.0:1789760763.844671:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774036427940 time 774121332306 state 0x0 ref 1 00000100:00100000:3.0:1789760763.844691:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774121592722 time 774121332306 state 0x1 ref 1 00000100:00100000:3.0:1789760763.844789:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774121592722 time 774185823829 state 0x0 ref 1 00000100:00100000:3.0:1789760763.844813:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493789312:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760763.845119:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25763840, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc807 00010000:00010000:3.0:1789760763.845130:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06228 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.845222:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:3.0:1789760763.845227:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.845235:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06228 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.845253:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06228 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760763.906847:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973609 is committed 00002000:00010000:3.0:1789760763.906875:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25763840 -> 25767936 00010000:00010000:3.0:1789760763.906900:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06228 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.906917:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06228 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.906929:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06228 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.906938:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760763.906945:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06228 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.906955:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06228 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.907029:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06228 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.907039:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06228 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.907873:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493789312:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63054us (65760us total) trans 4294973609 rc 0/0 00000100:00100000:3.0:1789760763.907888:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6320 00000100:00100000:3.0:1789760763.907897:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774121592722 time 774185823829 state 0x2 ref 0 00000100:00100000:2.0:1789760763.920067:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760763.920108:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493789440 00000100:00100000:3.0:1789760763.920119:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493789440 00000100:00100000:3.0:1789760763.920222:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774121592722 time 774185823829 state 0x0 ref 1 00000100:00100000:3.0:1789760763.920239:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774186084245 time 774185823829 state 0x1 ref 1 00000100:00100000:1.0:1789760763.920348:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774186084245 time 774261380899 state 0x0 ref 1 00000100:00100000:1.0:1789760763.920363:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493789440:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760763.920422:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25767936, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc807 00010000:00010000:1.0:1789760763.920431:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0622f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.920447:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760763.920452:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.920458:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0622f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.920472:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0622f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760763.983623:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25767936 -> 25772032 00010000:00010000:2.0:1789760763.983641:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0622f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.983654:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0622f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.983663:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0622f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.983675:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760763.983680:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0622f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.983689:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0622f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.983708:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0622f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.983716:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0622f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760763.983807:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493789440:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63443us (63744us total) trans 4294973610 rc 0/0 00000100:00100000:2.0:1789760763.983818:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6321 00000100:00100000:2.0:1789760763.983828:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774186084245 time 774261380899 state 0x2 ref 0 00000001:00080000:3.0:1789760763.983893:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973610 is committed 00000100:00100000:1.0:1789760763.990169:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760763.990265:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493789568 00000100:00100000:2.0:1789760763.990275:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493789568 00000100:00100000:2.0:1789760763.990348:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774186084245 time 774261380899 state 0x0 ref 1 00000100:00100000:2.0:1789760763.990364:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774261641315 time 774261380899 state 0x1 ref 1 00000100:00100000:2.0:1789760763.990395:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774261641315 time 774331430577 state 0x0 ref 1 00000100:00100000:2.0:1789760763.990411:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493789568:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760763.990457:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25772032, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc807 00010000:00010000:2.0:1789760763.990466:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06236 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.990482:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760763.990487:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760763.990493:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06236 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.990507:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06236 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.033953:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973611 is committed 00002000:00010000:2.0:1789760764.034151:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25772032 -> 25776128 00010000:00010000:2.0:1789760764.034167:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06236 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.034178:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06236 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.034186:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06236 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.034192:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760764.034197:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06236 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.034205:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06236 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.034223:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06236 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.034231:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06236 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760764.034332:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493789568:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43919us (44171us total) trans 4294973611 rc 0/0 00000100:00100000:2.0:1789760764.034343:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6322 00000100:00100000:2.0:1789760764.034352:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774261641315 time 774331430577 state 0x2 ref 0 00000100:00100000:1.0:1789760764.044752:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760764.044864:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493789696 00000100:00100000:2.0:1789760764.044874:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493789696 00000100:00100000:2.0:1789760764.044926:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774261641315 time 774331430577 state 0x0 ref 1 00000100:00100000:2.0:1789760764.044941:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774331690993 time 774331430577 state 0x1 ref 1 00000100:00100000:2.0:1789760764.044989:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774331690993 time 774386023696 state 0x0 ref 1 00000100:00100000:2.0:1789760764.045002:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493789696:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760764.045049:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25776128, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc807 00010000:00010000:2.0:1789760764.045058:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0623d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.045074:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760764.045079:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.045086:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0623d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.045101:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0623d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.094200:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973612 is committed 00002000:00010000:3.0:1789760764.094787:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25776128 -> 25780224 00010000:00010000:3.0:1789760764.094802:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0623d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.094814:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0623d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.094822:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0623d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.094828:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760764.094832:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0623d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.094840:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0623d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.094859:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0623d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.094866:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e0623d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.095037:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493789696:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50032us (50289us total) trans 4294973612 rc 0/0 00000100:00100000:3.0:1789760764.095052:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6323 00000100:00100000:3.0:1789760764.095537:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774331690993 time 774386023696 state 0x2 ref 0 00000100:00100000:0.0:1789760764.100149:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760764.100322:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493789824 00000100:00100000:3.0:1789760764.100331:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493789824 00000100:00100000:3.0:1789760764.100392:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774331690993 time 774386023696 state 0x0 ref 1 00000100:00100000:3.0:1789760764.100407:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774386284112 time 774386023696 state 0x1 ref 1 00000100:00100000:3.0:1789760764.100476:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774386284112 time 774441510935 state 0x0 ref 1 00000100:00100000:3.0:1789760764.100489:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493789824:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760764.100535:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25780224, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc807 00010000:00010000:3.0:1789760764.100544:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06244 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.100560:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:3.0:1789760764.100565:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.100570:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06244 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.100594:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06244 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760764.142281:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25780224 -> 25784320 00010000:00010000:3.0:1789760764.142301:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06244 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.142325:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06244 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.142333:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06244 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.142340:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760764.142344:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06244 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.142353:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06244 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.142382:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06244 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.142390:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06244 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.142467:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493789824:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41975us (42323us total) trans 4294973613 rc 0/0 00000100:00100000:3.0:1789760764.142476:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6324 00000100:00100000:3.0:1789760764.142486:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774386284112 time 774441510935 state 0x2 ref 0 00000001:00080000:2.0:1789760764.144572:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973613 is committed 00000100:00100000:1.0:1789760764.147592:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760764.147703:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493789952 00000100:00100000:2.0:1789760764.147713:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493789952 00000100:00100000:2.0:1789760764.147770:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774386284112 time 774441510935 state 0x0 ref 1 00000100:00100000:2.0:1789760764.147786:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774441771351 time 774441510935 state 0x1 ref 1 00000100:00100000:2.0:1789760764.147929:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774441771351 time 774488962777 state 0x0 ref 1 00000100:00100000:2.0:1789760764.147944:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493789952:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760764.148072:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25784320, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc807 00010000:00010000:2.0:1789760764.148081:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0624b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.148097:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760764.148101:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.148108:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0624b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.148124:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0624b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760764.171196:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973614 is committed 00002000:00010000:2.0:1789760764.171256:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25784320 -> 25788416 00010000:00010000:2.0:1789760764.171271:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0624b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.171284:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0624b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.171293:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0624b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.171301:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760764.171306:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0624b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.171326:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0624b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.171344:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0624b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.171352:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0624b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760764.171444:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493789952:12345-192.168.203.60@tcp:4:dd.0 Request processed in 23499us (23859us total) trans 4294973614 rc 0/0 00000100:00100000:2.0:1789760764.171454:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6325 00000100:00100000:2.0:1789760764.171464:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774441771351 time 774488962777 state 0x2 ref 0 00000100:00100000:1.0:1789760764.175552:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760764.175639:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493790080 00000100:00100000:2.0:1789760764.175649:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493790080 00000100:00100000:2.0:1789760764.175713:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774441771351 time 774488962777 state 0x0 ref 1 00000100:00100000:2.0:1789760764.175730:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774489223193 time 774488962777 state 0x1 ref 1 00000100:00100000:2.0:1789760764.175760:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774489223193 time 774516795034 state 0x0 ref 1 00000100:00100000:2.0:1789760764.175773:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493790080:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760764.175838:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25788416, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc807 00010000:00010000:2.0:1789760764.175848:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06252 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.175865:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760764.175870:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.175876:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06252 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.175890:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06252 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760764.225420:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25788416 -> 25792512 00010000:00010000:0.0:1789760764.225440:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06252 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.225453:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06252 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.225462:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06252 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.225469:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760764.225473:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06252 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.225483:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06252 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.225502:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06252 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.225510:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06252 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.225599:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493790080:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49824us (50053us total) trans 4294973615 rc 0/0 00000100:00100000:0.0:1789760764.225610:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6326 00000100:00100000:0.0:1789760764.225620:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774489223193 time 774516795034 state 0x2 ref 0 00000001:00080000:2.0:1789760764.226754:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973615 is committed 00000100:00100000:0.0:1789760764.229611:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760764.229741:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493790208 00000100:00100000:2.0:1789760764.229750:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493790208 00000100:00100000:2.0:1789760764.229915:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774489223193 time 774516795034 state 0x0 ref 1 00000100:00100000:2.0:1789760764.229930:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774517055450 time 774516795034 state 0x1 ref 1 00000100:00100000:2.0:1789760764.230047:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774517055450 time 774570997026 state 0x0 ref 1 00000100:00100000:2.0:1789760764.230062:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493790208:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760764.230108:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25792512, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc807 00010000:00010000:2.0:1789760764.230118:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06259 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.230132:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760764.230137:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.230143:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06259 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.230158:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06259 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760764.273570:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973616 is committed 00002000:00010000:2.0:1789760764.273866:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25792512 -> 25796608 00010000:00010000:2.0:1789760764.273881:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06259 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.273899:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06259 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.273907:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06259 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.273913:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760764.273918:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06259 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.273926:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06259 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.273945:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06259 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.273953:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06259 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760764.274117:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493790208:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44053us (44510us total) trans 4294973616 rc 0/0 00000100:00100000:2.0:1789760764.274129:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6327 00000100:00100000:2.0:1789760764.274139:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774517055450 time 774570997026 state 0x2 ref 0 00000100:00100000:1.0:1789760764.277417:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760764.277484:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493790336 00000100:00100000:3.0:1789760764.277493:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493790336 00000100:00100000:3.0:1789760764.277537:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774517055450 time 774570997026 state 0x0 ref 1 00000100:00100000:3.0:1789760764.277548:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774571257442 time 774570997026 state 0x1 ref 1 00000100:00100000:3.0:1789760764.277605:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774571257442 time 774618640488 state 0x0 ref 1 00000100:00100000:3.0:1789760764.277618:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493790336:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760764.277664:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25796608, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc807 00010000:00010000:3.0:1789760764.277673:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06260 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.277688:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:3.0:1789760764.277693:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.277699:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06260 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.277712:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06260 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760764.283630:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc452d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3304:1876699519257088:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.283651:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc47ac380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3306:1876699519257344:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.283752:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760764.283755:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc47aed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3303:1876699519257472:0@lo:400:kworker.0 00000100:00100000:1.0:1789760764.283784:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760764.283807:0:3306:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789760764.283814:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc452c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3306:1876699519257216:0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.283836:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760764.283902:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760764.283984:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519257088 00000100:00100000:3.0:1789760764.283991:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519257088 00000100:00100000:3.0:1789760764.284034:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 408 00000100:00100000:3.0:1789760764.284041:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d48050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3304:x1876699519257088:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.284091:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d48050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3304:x1876699519257088:12345-0@lo:400:kworker.0 Request processed in 45us (185us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760764.284099:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 408 00000100:00100000:1.0:1789760764.284109:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519257344 00000100:00100000:1.0:1789760764.284114:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519257344 00000100:00100000:1.0:1789760764.284158:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519257472 00000100:00100000:1.0:1789760764.284162:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519257472 00000100:00080000:0.0:1789760764.284179:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760764.284187:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519257216 00000100:00100000:1.0:1789760764.284191:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519257216 00000100:00100000:1.0:1789760764.284207:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 504 00000100:00100000:1.0:1789760764.284216:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3306:x1876699519257344:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760764.284266:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3306:x1876699519257344:12345-0@lo:400:kworker.0 Request processed in 52us (513us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760764.284273:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 504 00000100:00100000:1.0:1789760764.284291:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 505 00000100:00100000:1.0:1789760764.284299:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9621018e7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876699519257472:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760764.284350:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9621018e7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876699519257472:12345-0@lo:400:kworker.0 Request processed in 53us (564us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760764.284356:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 505 00000100:00100000:1.0:1789760764.284369:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 506 00000100:00100000:1.0:1789760764.284375:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8555180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3306:x1876699519257216:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760764.284401:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8555180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3306:x1876699519257216:12345-0@lo:400:kworker.0 Request processed in 26us (564us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760764.284408:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 506 00000100:00100000:0.0:1789760764.284442:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc47aed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3303:1876699519257472:0@lo:400:kworker.0 00000100:00100000:1.0:1789760764.284449:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc452c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3306:1876699519257216:0@lo:400:kworker.0 00000100:00100000:1.0:1789760764.284462:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc47ac380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3306:1876699519257344:0@lo:400:kworker.0 00000100:00100000:2.0:1789760764.284607:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc452d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3304:1876699519257088:0@lo:400:kworker.0 00000100:00100000:2.0:1789760764.284616:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620c8555880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3304:1876699519257600:0@lo:400:kworker.0 00000100:00100000:2.0:1789760764.284648:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760764.284758:0:6689:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519257600 00000100:00100000:3.0:1789760764.284763:0:6689:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519257600 00000100:00080000:1.0:1789760764.284808:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789760764.284809:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 182 00000100:00100000:3.0:1789760764.284817:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519257600:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760764.284859:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519257600:12345-0@lo:400:kworker.0 Request processed in 43us (209us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760764.284865:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 182 00000100:00100000:1.0:1789760764.284931:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620c8555880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3304:1876699519257600:0@lo:400:kworker.0 00000001:00080000:2.0:1789760764.310433:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973617 is committed 00002000:00010000:1.0:1789760764.312431:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25796608 -> 25800704 00010000:00010000:1.0:1789760764.312452:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06260 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.312465:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06260 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.312474:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06260 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.312481:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760764.312486:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06260 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.312495:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06260 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.312522:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06260 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.312529:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06260 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.312635:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493790336:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35013us (35219us total) trans 4294973617 rc 0/0 00000100:00100000:1.0:1789760764.312646:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6328 00000100:00100000:1.0:1789760764.312656:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774571257442 time 774618640488 state 0x2 ref 0 00000100:00100000:1.0:1789760764.315569:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760764.315661:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493790464 00000100:00100000:0.0:1789760764.315670:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493790464 00000100:00100000:0.0:1789760764.315725:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774571257442 time 774618640488 state 0x0 ref 1 00000100:00100000:0.0:1789760764.315741:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774618900904 time 774618640488 state 0x1 ref 1 00000100:00100000:0.0:1789760764.315774:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774618900904 time 774656809325 state 0x0 ref 1 00000100:00100000:0.0:1789760764.315785:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493790464:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760764.315832:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25800704, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc807 00010000:00010000:0.0:1789760764.315841:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06267 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.315855:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:0.0:1789760764.315861:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.315867:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06267 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.315880:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06267 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760764.357922:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973618 is committed 00002000:00010000:0.0:1789760764.358361:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25800704 -> 25804800 00010000:00010000:0.0:1789760764.358375:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06267 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.358388:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06267 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.358402:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06267 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.358409:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760764.358414:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06267 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.358423:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06267 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.358443:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06267 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.358450:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06267 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.358583:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493790464:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42794us (43019us total) trans 4294973618 rc 0/0 00000100:00100000:0.0:1789760764.358595:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6329 00000100:00100000:0.0:1789760764.358605:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774618900904 time 774656809325 state 0x2 ref 0 00000100:00100000:0.0:1789760764.362717:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760764.362808:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493790592 00000100:00100000:2.0:1789760764.362860:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493790592 00000100:00100000:2.0:1789760764.362918:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774618900904 time 774656809325 state 0x0 ref 1 00000100:00100000:2.0:1789760764.362933:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774657069741 time 774656809325 state 0x1 ref 1 00000100:00100000:1.0:1789760764.363009:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774657069741 time 774704043142 state 0x0 ref 1 00000100:00100000:1.0:1789760764.363025:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493790592:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760764.363081:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25804800, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc807 00010000:00010000:1.0:1789760764.363090:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0626e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.363107:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760764.363112:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760764.363118:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0626e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.363132:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0626e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.417919:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973619 is committed 00002000:00010000:2.0:1789760764.418019:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25804800 -> 25808896 00010000:00010000:2.0:1789760764.418035:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0626e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.418047:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0626e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.418056:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0626e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.418063:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760764.418068:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0626e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.418077:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0626e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.418101:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0626e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.418109:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0626e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760764.418236:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493790592:12345-192.168.203.60@tcp:4:dd.0 Request processed in 55210us (55524us total) trans 4294973619 rc 0/0 00000100:00100000:2.0:1789760764.418247:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6330 00000100:00100000:2.0:1789760764.418257:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774657069741 time 774704043142 state 0x2 ref 0 00000100:00100000:1.0:1789760764.425817:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760764.425946:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493790720 00000100:00100000:2.0:1789760764.425956:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493790720 00000100:00100000:2.0:1789760764.426044:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774657069741 time 774704043142 state 0x0 ref 1 00000100:00100000:2.0:1789760764.426059:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774704303558 time 774704043142 state 0x1 ref 1 00000100:00100000:2.0:1789760764.426080:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774704303558 time 774767116138 state 0x0 ref 1 00000100:00100000:2.0:1789760764.426091:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493790720:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760764.426143:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25808896, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc807 00010000:00010000:2.0:1789760764.426153:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06275 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.426168:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760764.426174:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.426181:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06275 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.426193:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06275 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760764.463116:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973620 is committed 00002000:00010000:0.0:1789760764.463279:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25808896 -> 25812992 00010000:00010000:0.0:1789760764.463296:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06275 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.463319:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06275 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.463329:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06275 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.463337:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760764.463342:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06275 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.463352:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06275 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.463440:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06275 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.463450:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06275 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.463580:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493790720:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37486us (37768us total) trans 4294973620 rc 0/0 00000100:00100000:0.0:1789760764.463591:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6331 00000100:00100000:0.0:1789760764.463601:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774704303558 time 774767116138 state 0x2 ref 0 00000100:00100000:0.0:1789760764.467527:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760764.467592:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493790848 00000100:00100000:2.0:1789760764.467601:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493790848 00000100:00100000:2.0:1789760764.467661:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774704303558 time 774767116138 state 0x0 ref 1 00000100:00100000:2.0:1789760764.467677:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774767376554 time 774767116138 state 0x1 ref 1 00000100:00100000:2.0:1789760764.467765:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774767376554 time 774808799361 state 0x0 ref 1 00000100:00100000:2.0:1789760764.467778:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493790848:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760764.467829:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25812992, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc807 00010000:00010000:2.0:1789760764.467837:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0627c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.467864:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760764.467869:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.467876:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0627c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.467891:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0627c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.474134:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760764.502340:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973621 is committed 00002000:00010000:3.0:1789760764.502590:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25812992 -> 25817088 00010000:00010000:3.0:1789760764.502606:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0627c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.502618:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0627c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.502627:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0627c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.502635:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760764.502639:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0627c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.502653:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0627c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.502672:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0627c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.502680:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0627c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.502800:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493790848:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35019us (35276us total) trans 4294973621 rc 0/0 00000100:00100000:3.0:1789760764.502810:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6332 00000100:00100000:3.0:1789760764.502820:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774767376554 time 774808799361 state 0x2 ref 0 00000100:00100000:2.0:1789760764.507679:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760764.507815:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493790976 00000100:00100000:0.0:1789760764.507826:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493790976 00000100:00100000:0.0:1789760764.507879:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774767376554 time 774808799361 state 0x0 ref 1 00000100:00100000:0.0:1789760764.507893:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774809059777 time 774808799361 state 0x1 ref 1 00000100:00100000:0.0:1789760764.507923:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774809059777 time 774848958475 state 0x0 ref 1 00000100:00100000:0.0:1789760764.507936:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493790976:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760764.508018:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25817088, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc807 00010000:00010000:0.0:1789760764.508027:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06283 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.508044:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:0.0:1789760764.508048:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.508055:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06283 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.508069:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06283 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760764.546035:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973622 is committed 00002000:00010000:0.0:1789760764.546223:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25817088 -> 25821184 00010000:00010000:0.0:1789760764.546240:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06283 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.546328:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06283 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.546339:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06283 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.546347:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760764.546353:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06283 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.546373:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06283 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.546391:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06283 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.546399:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06283 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.546498:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493790976:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38559us (38822us total) trans 4294973622 rc 0/0 00000100:00100000:0.0:1789760764.546509:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6333 00000100:00100000:0.0:1789760764.546519:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774809059777 time 774848958475 state 0x2 ref 0 00000100:00100000:1.0:1789760764.552071:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760764.553363:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493791104 00000100:00100000:2.0:1789760764.553376:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493791104 00000100:00100000:2.0:1789760764.553442:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774809059777 time 774848958475 state 0x0 ref 1 00000100:00100000:2.0:1789760764.553459:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774849218891 time 774848958475 state 0x1 ref 1 00000100:00100000:2.0:1789760764.553492:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774849218891 time 774894527401 state 0x0 ref 1 00000100:00100000:2.0:1789760764.553519:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620f39f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493791104:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760764.553571:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25821184, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc807 00010000:00010000:2.0:1789760764.553579:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0628a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.553595:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760764.553600:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.553606:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0628a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.553620:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0628a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760764.588669:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973623 is committed 00002000:00010000:2.0:1789760764.588779:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25821184 -> 25825280 00010000:00010000:2.0:1789760764.588862:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0628a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.588875:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0628a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.588884:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0628a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.588892:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760764.588897:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0628a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.588907:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0628a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.588927:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0628a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.588935:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0628a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760764.589054:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620f39f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493791104:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35546us (36987us total) trans 4294973623 rc 0/0 00000100:00100000:2.0:1789760764.589064:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6334 00000100:00100000:2.0:1789760764.589074:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774849218891 time 774894527401 state 0x2 ref 0 00000100:00100000:2.0:1789760764.593735:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760764.593931:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493791232 00000100:00100000:0.0:1789760764.593941:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493791232 00000100:00100000:0.0:1789760764.594504:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774849218891 time 774894527401 state 0x0 ref 1 00000100:00100000:0.0:1789760764.594519:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774894787817 time 774894527401 state 0x1 ref 1 00000100:00100000:0.0:1789760764.594657:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774894787817 time 774935691608 state 0x0 ref 1 00000100:00100000:0.0:1789760764.594672:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493791232:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760764.594721:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25825280, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xc807 00010000:00010000:0.0:1789760764.594731:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06291 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.594746:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:0.0:1789760764.594751:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.594757:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06291 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.594770:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06291 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760764.626288:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25825280 -> 25829376 00002000:00010000:2.0:1789760764.626297:0:13038:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760763 -> 1789760764 00002000:00010000:2.0:1789760764.626300:0:13038:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760763 -> 1789760764 00010000:00010000:2.0:1789760764.626320:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06291 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.626333:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06291 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.626343:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06291 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.626350:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760764.626355:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06291 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.626486:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06291 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.626506:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06291 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.626514:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06291 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760764.626620:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493791232:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31944us (32888us total) trans 4294973624 rc 0/0 00000100:00100000:2.0:1789760764.626633:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6335 00000100:00100000:2.0:1789760764.626642:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774894787817 time 774935691608 state 0x2 ref 0 00000001:00080000:3.0:1789760764.626783:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973624 is committed 00000100:00100000:2.0:1789760764.631470:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760764.631619:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493791360 00000100:00100000:3.0:1789760764.631628:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493791360 00000100:00100000:3.0:1789760764.631682:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774894787817 time 774935691608 state 0x0 ref 1 00000100:00100000:3.0:1789760764.631697:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774935952024 time 774935691608 state 0x1 ref 1 00000100:00100000:3.0:1789760764.631724:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774935952024 time 774972759619 state 0x0 ref 1 00000100:00100000:3.0:1789760764.631736:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c89fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493791360:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760764.631787:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25829376, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc807 00010000:00010000:3.0:1789760764.631806:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06298 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.631821:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=774s v=(0 0) 00010000:00010000:3.0:1789760764.631829:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.631835:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06298 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.631849:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06298 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.673606:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973625 is committed 00002000:00010000:2.0:1789760764.673756:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25829376 -> 25833472 00010000:00010000:2.0:1789760764.673770:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06298 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.673782:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06298 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.673860:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06298 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.673868:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760764.673873:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06298 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.673881:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06298 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.673901:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06298 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.673909:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06298 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760764.674111:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c89fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493791360:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42372us (42648us total) trans 4294973625 rc 0/0 00000100:00100000:2.0:1789760764.674121:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6336 00000100:00100000:2.0:1789760764.674131:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774935952024 time 774972759619 state 0x2 ref 0 00000100:00100000:0.0:1789760764.682143:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760764.682252:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493791488 00000100:00100000:2.0:1789760764.682263:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493791488 00000100:00100000:2.0:1789760764.682345:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774935952024 time 774972759619 state 0x0 ref 1 00000100:00100000:2.0:1789760764.682371:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774973020035 time 774972759619 state 0x1 ref 1 00000100:00100000:2.0:1789760764.682403:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774973020035 time 775023438309 state 0x0 ref 1 00000100:00100000:2.0:1789760764.682415:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493791488:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760764.682463:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25833472, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc807 00010000:00010000:2.0:1789760764.682472:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0629f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.682486:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760764.682490:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.682496:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0629f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.682511:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0629f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760764.736247:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25833472 -> 25837568 00010000:00010000:0.0:1789760764.736268:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0629f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.736282:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0629f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.736291:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0629f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.736298:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760764.736313:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0629f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.736322:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0629f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.736341:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0629f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.736350:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0629f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.736459:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493791488:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54041us (54320us total) trans 4294973626 rc 0/0 00000100:00100000:0.0:1789760764.736469:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6337 00000100:00100000:0.0:1789760764.736478:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774973020035 time 775023438309 state 0x2 ref 0 00000001:00080000:1.0:1789760764.736608:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973626 is committed 00000100:00100000:0.0:1789760764.740406:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760764.740520:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493791616 00000100:00100000:2.0:1789760764.740530:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493791616 00000100:00100000:2.0:1789760764.740585:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 774973020035 time 775023438309 state 0x0 ref 1 00000100:00100000:2.0:1789760764.740600:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775023698725 time 775023438309 state 0x1 ref 1 00000100:00100000:2.0:1789760764.740633:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775023698725 time 775081667943 state 0x0 ref 1 00000100:00100000:2.0:1789760764.740646:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493791616:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760764.740771:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25837568, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc807 00010000:00010000:2.0:1789760764.740781:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e062a6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.740805:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760764.740809:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.740816:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e062a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.740830:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e062a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.794999:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620c8554e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519257728:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760764.795066:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760764.795094:0:6691:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519257728 00000100:00100000:2.0:1789760764.795102:0:6691:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519257728 00000100:00100000:2.0:1789760764.795146:0:6691:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 209 00000100:00100000:2.0:1789760764.795155:0:6691:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8556a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519257728:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760764.795191:0:6691:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760764.795330:0:6691:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8556a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519257728:12345-0@lo:13:osp-pre-1-0.0 Request processed in 175us (264us total) trans 0 rc 0/0 00000001:00080000:1.0:1789760764.800337:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973627 is committed 00002000:00010000:0.0:1789760764.800475:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25837568 -> 25841664 00010000:00010000:0.0:1789760764.800492:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e062a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.800505:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e062a6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.800514:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e062a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.800525:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760764.800529:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e062a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.800539:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e062a6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.800558:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e062a6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.800571:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e062a6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.800657:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493791616:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60010us (60259us total) trans 4294973627 rc 0/0 00000100:00100000:0.0:1789760764.802168:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6338 00000100:00100000:0.0:1789760764.802178:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775023698725 time 775081667943 state 0x2 ref 0 00000100:00100000:3.0:1789760764.802628:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620c8554e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519257728:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760764.802704:0:6691:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 209 00000100:00100000:1.0:1789760764.805223:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760764.805313:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493791744 00000100:00100000:0.0:1789760764.805323:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493791744 00000100:00100000:0.0:1789760764.805380:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775023698725 time 775081667943 state 0x0 ref 1 00000100:00100000:0.0:1789760764.805395:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775081928359 time 775081667943 state 0x1 ref 1 00000100:00100000:0.0:1789760764.805465:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775081928359 time 775146499061 state 0x0 ref 1 00000100:00100000:0.0:1789760764.805478:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493791744:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760764.805531:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25841664, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc807 00010000:00010000:0.0:1789760764.805540:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e062ad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.805557:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760764.805562:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.805568:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e062ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.805588:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e062ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760764.863955:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25841664 -> 25845760 00010000:00010000:1.0:1789760764.864002:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e062ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.864015:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e062ad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.864023:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973628 is committed 00010000:00010000:1.0:1789760764.864033:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e062ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.864041:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760764.864045:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e062ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.864054:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e062ad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.864074:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e062ad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.864082:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e062ad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.864185:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493791744:12345-192.168.203.60@tcp:4:dd.0 Request processed in 58703us (58965us total) trans 4294973628 rc 0/0 00000100:00100000:1.0:1789760764.864326:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6339 00000100:00100000:1.0:1789760764.864337:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775081928359 time 775146499061 state 0x2 ref 0 00000100:00100000:1.0:1789760764.870554:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760764.870807:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493791872 00000100:00100000:3.0:1789760764.870818:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493791872 00000100:00100000:3.0:1789760764.870875:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775081928359 time 775146499061 state 0x0 ref 1 00000100:00100000:3.0:1789760764.870889:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775146759477 time 775146499061 state 0x1 ref 1 00000100:00100000:3.0:1789760764.870918:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775146759477 time 775211954298 state 0x0 ref 1 00000100:00100000:3.0:1789760764.870931:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493791872:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760764.870994:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25845760, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc807 00010000:00010000:3.0:1789760764.871003:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e062b4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.871020:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760764.871111:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.871117:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e062b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.871140:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e062b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.926574:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973629 is committed 00002000:00010000:2.0:1789760764.927559:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25845760 -> 25849856 00010000:00010000:2.0:1789760764.927576:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e062b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.927588:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e062b4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.927597:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e062b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.927604:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760764.927608:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e062b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.927618:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e062b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.927637:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e062b4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.927645:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e062b4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760764.927748:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493791872:12345-192.168.203.60@tcp:4:dd.0 Request processed in 56813us (57200us total) trans 4294973629 rc 0/0 00000100:00100000:2.0:1789760764.927759:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6340 00000100:00100000:2.0:1789760764.927768:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775146759477 time 775211954298 state 0x2 ref 0 00000100:00100000:2.0:1789760764.931635:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760764.931871:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493792000 00000100:00100000:3.0:1789760764.931882:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493792000 00000100:00100000:3.0:1789760764.931937:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775146759477 time 775211954298 state 0x0 ref 1 00000100:00100000:3.0:1789760764.931952:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775212214714 time 775211954298 state 0x1 ref 1 00000100:00100000:3.0:1789760764.932060:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775212214714 time 775273094444 state 0x0 ref 1 00000100:00100000:3.0:1789760764.932073:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493792000:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760764.932119:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25849856, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc807 00010000:00010000:3.0:1789760764.932128:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e062bb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.932142:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760764.932147:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.932153:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e062bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.932167:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e062bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760764.960819:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973630 is committed 00002000:00010000:1.0:1789760764.961022:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25849856 -> 25853952 00010000:00010000:1.0:1789760764.961037:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e062bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.961050:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e062bb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.961096:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e062bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.961105:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760764.961110:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e062bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.961119:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e062bb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.961143:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e062bb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.961152:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e062bb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.961244:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493792000:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29168us (29612us total) trans 4294973630 rc 0/0 00000100:00100000:1.0:1789760764.961255:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6341 00000100:00100000:1.0:1789760764.961264:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775212214714 time 775273094444 state 0x2 ref 0 00000100:00100000:1.0:1789760764.965404:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760764.965486:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493792128 00000100:00100000:0.0:1789760764.965497:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493792128 00000100:00100000:0.0:1789760764.965554:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775212214714 time 775273094444 state 0x0 ref 1 00000100:00100000:0.0:1789760764.965570:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775273354860 time 775273094444 state 0x1 ref 1 00000100:00100000:0.0:1789760764.965604:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775273354860 time 775306639693 state 0x0 ref 1 00000100:00100000:0.0:1789760764.965618:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d8b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493792128:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760764.965667:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25853952, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc807 00010000:00010000:0.0:1789760764.965676:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e062c2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.965693:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760764.965698:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.965703:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e062c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.965718:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e062c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760764.999985:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25853952 -> 25858048 00010000:00010000:0.0:1789760765.000006:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e062c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.000019:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e062c2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.000028:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e062c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.000036:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760765.000041:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e062c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.000050:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e062c2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.000070:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e062c2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.000078:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e062c2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.000174:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d8b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493792128:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34553us (34775us total) trans 4294973631 rc 0/0 00000100:00100000:0.0:1789760765.000189:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6342 00000100:00100000:0.0:1789760765.000198:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775273354860 time 775306639693 state 0x2 ref 0 00000001:00080000:2.0:1789760765.000650:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973631 is committed 00000100:00100000:0.0:1789760765.003250:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760765.003345:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493792256 00000100:00100000:1.0:1789760765.003354:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493792256 00000100:00100000:1.0:1789760765.003486:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775273354860 time 775306639693 state 0x0 ref 1 00000100:00100000:1.0:1789760765.003501:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775306900109 time 775306639693 state 0x1 ref 1 00000100:00100000:1.0:1789760765.003561:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775306900109 time 775344596204 state 0x0 ref 1 00000100:00100000:1.0:1789760765.003575:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493792256:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760765.003616:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25858048, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc807 00010000:00010000:1.0:1789760765.003625:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e062c9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.003646:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:1.0:1789760765.003651:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760765.003657:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e062c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.003671:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e062c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760765.004201:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789760765.029027:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25858048 -> 25862144 00010000:00010000:1.0:1789760765.029049:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e062c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.029062:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e062c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.029072:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e062c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.029080:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760765.029085:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e062c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.029095:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e062c9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.029115:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e062c9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.029124:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e062c9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.029209:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493792256:12345-192.168.203.60@tcp:4:dd.0 Request processed in 25631us (25961us total) trans 4294973632 rc 0/0 00000100:00100000:1.0:1789760765.029220:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6343 00000100:00100000:1.0:1789760765.029230:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775306900109 time 775344596204 state 0x2 ref 0 00000001:00080000:2.0:1789760765.029702:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973632 is committed 00000100:00100000:1.0:1789760765.034056:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760765.034264:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493792384 00000100:00100000:2.0:1789760765.034273:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493792384 00000100:00100000:2.0:1789760765.034340:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775306900109 time 775344596204 state 0x0 ref 1 00000100:00100000:2.0:1789760765.034355:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775344856620 time 775344596204 state 0x1 ref 1 00000100:00100000:2.0:1789760765.034385:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775344856620 time 775375420498 state 0x0 ref 1 00000100:00100000:2.0:1789760765.034398:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb452a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493792384:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760765.034445:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25862144, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc807 00010000:00010000:2.0:1789760765.034454:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e062d0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.034468:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760765.034472:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.034478:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e062d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.034492:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e062d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760765.081568:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973633 is committed 00002000:00010000:1.0:1789760765.081715:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25862144 -> 25866240 00010000:00010000:1.0:1789760765.081731:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e062d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.081744:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e062d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.081753:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e062d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.081761:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760765.081765:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e062d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.081774:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e062d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.081808:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e062d0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.081817:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e062d0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.081906:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb452a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493792384:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47505us (47856us total) trans 4294973633 rc 0/0 00000100:00100000:1.0:1789760765.081917:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6344 00000100:00100000:1.0:1789760765.081927:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775344856620 time 775375420498 state 0x2 ref 0 00000100:00100000:1.0:1789760765.088859:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760765.088996:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493792512 00000100:00100000:3.0:1789760765.089006:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493792512 00000100:00100000:3.0:1789760765.089062:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775344856620 time 775375420498 state 0x0 ref 1 00000100:00100000:3.0:1789760765.089076:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775375680914 time 775375420498 state 0x1 ref 1 00000100:00100000:3.0:1789760765.089106:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775375680914 time 775430141474 state 0x0 ref 1 00000100:00100000:3.0:1789760765.089118:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb453800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493792512:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760765.089165:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25866240, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc807 00010000:00010000:3.0:1789760765.089174:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e062d7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.089190:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760765.089194:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.089201:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e062d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.089215:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e062d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760765.151650:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973634 is committed 00002000:00010000:3.0:1789760765.151900:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25866240 -> 25870336 00010000:00010000:3.0:1789760765.152125:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e062d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.152140:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e062d7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.152149:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e062d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.152157:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760765.152162:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e062d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.152172:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e062d7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.152193:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e062d7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.152202:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e062d7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.152367:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb453800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493792512:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63238us (63503us total) trans 4294973634 rc 0/0 00000100:00100000:3.0:1789760765.152378:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6345 00000100:00100000:3.0:1789760765.152389:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775375680914 time 775430141474 state 0x2 ref 0 00000100:00100000:0.0:1789760765.157708:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760765.157924:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493792640 00000100:00100000:2.0:1789760765.157934:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493792640 00000100:00100000:2.0:1789760765.158522:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775375680914 time 775430141474 state 0x0 ref 1 00000100:00100000:2.0:1789760765.158539:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775430401890 time 775430141474 state 0x1 ref 1 00000100:00100000:2.0:1789760765.158623:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775430401890 time 775499657322 state 0x0 ref 1 00000100:00100000:2.0:1789760765.158639:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493792640:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760765.158690:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25870336, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc807 00010000:00010000:2.0:1789760765.158700:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e062de lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.158716:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760765.158721:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.158729:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e062de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.158743:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e062de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760765.181512:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973635 is committed 00002000:00010000:2.0:1789760765.181609:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25870336 -> 25874432 00010000:00010000:2.0:1789760765.181624:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e062de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.181636:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e062de lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.181642:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e062de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.181647:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760765.181650:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e062de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.181656:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e062de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.181669:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e062de lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.181673:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e062de lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760765.181750:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493792640:12345-192.168.203.60@tcp:4:dd.0 Request processed in 23111us (24047us total) trans 4294973635 rc 0/0 00000100:00100000:2.0:1789760765.181765:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6346 00000100:00100000:2.0:1789760765.181774:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775430401890 time 775499657322 state 0x2 ref 0 00000100:00100000:1.0:1789760765.184227:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760765.184333:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493792768 00000100:00100000:2.0:1789760765.184342:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493792768 00000100:00100000:2.0:1789760765.184401:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775430401890 time 775499657322 state 0x0 ref 1 00000100:00100000:2.0:1789760765.184414:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775499917738 time 775499657322 state 0x1 ref 1 00000100:00100000:2.0:1789760765.184441:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775499917738 time 775525476794 state 0x0 ref 1 00000100:00100000:2.0:1789760765.184453:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc352c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493792768:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760765.184498:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25874432, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc807 00010000:00010000:2.0:1789760765.184505:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e062e5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.184520:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760765.184524:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.184530:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e062e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.184543:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e062e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760765.211873:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973636 is committed 00002000:00010000:0.0:1789760765.212073:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25874432 -> 25878528 00010000:00010000:0.0:1789760765.212089:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e062e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.212229:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e062e5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.212237:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e062e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.212241:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760765.212245:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e062e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.212251:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e062e5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.212266:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e062e5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.212275:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e062e5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.212384:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493792768:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27927us (28160us total) trans 4294973636 rc 0/0 00000100:00100000:0.0:1789760765.212394:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6347 00000100:00100000:0.0:1789760765.212404:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775499917738 time 775525476794 state 0x2 ref 0 00000100:00100000:1.0:1789760765.217227:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760765.217402:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493792896 00000100:00100000:2.0:1789760765.217412:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493792896 00000100:00100000:2.0:1789760765.217469:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775499917738 time 775525476794 state 0x0 ref 1 00000100:00100000:2.0:1789760765.217484:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775525737210 time 775525476794 state 0x1 ref 1 00000100:00100000:2.0:1789760765.217549:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775525737210 time 775558584260 state 0x0 ref 1 00000100:00100000:2.0:1789760765.217563:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4badc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493792896:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760765.217608:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25878528, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc807 00010000:00010000:2.0:1789760765.217617:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e062ec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.217633:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760765.217637:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.217643:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e062ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.217658:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e062ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760765.257831:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973637 is committed 00002000:00010000:0.0:1789760765.257976:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25878528 -> 25882624 00010000:00010000:0.0:1789760765.257992:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e062ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.258005:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e062ec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.258014:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e062ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.258021:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760765.258026:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e062ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.258035:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e062ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.258060:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e062ec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.258068:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e062ec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.258174:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4badc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493792896:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40608us (40951us total) trans 4294973637 rc 0/0 00000100:00100000:0.0:1789760765.258185:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6348 00000100:00100000:0.0:1789760765.258196:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775525737210 time 775558584260 state 0x2 ref 0 00000100:00100000:1.0:1789760765.262519:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760765.262698:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493793024 00000100:00100000:3.0:1789760765.262711:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493793024 00000100:00100000:3.0:1789760765.262766:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775525737210 time 775558584260 state 0x0 ref 1 00000100:00100000:3.0:1789760765.262781:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775558844676 time 775558584260 state 0x1 ref 1 00000100:00100000:3.0:1789760765.262808:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775558844676 time 775603843488 state 0x0 ref 1 00000100:00100000:3.0:1789760765.262820:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493793024:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760765.262994:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25882624, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc807 00010000:00010000:3.0:1789760765.263006:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e062f3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.263021:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760765.263026:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.263031:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e062f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.263045:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e062f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760765.299774:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25882624 -> 25886720 00010000:00010000:0.0:1789760765.299795:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e062f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.299809:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e062f3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760765.300336:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973638 is committed 00010000:00010000:0.0:1789760765.300439:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e062f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.300447:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760765.300451:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e062f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.300467:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e062f3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.300488:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e062f3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.300496:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e062f3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.300629:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493793024:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37805us (38114us total) trans 4294973638 rc 0/0 00000100:00100000:0.0:1789760765.300641:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6349 00000100:00100000:0.0:1789760765.300651:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775558844676 time 775603843488 state 0x2 ref 0 00000100:00100000:3.0:1789760765.305835:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760765.305906:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493793152 00000100:00100000:0.0:1789760765.305918:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493793152 00000100:00100000:0.0:1789760765.305993:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775558844676 time 775603843488 state 0x0 ref 1 00000100:00100000:0.0:1789760765.306009:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775604103904 time 775603843488 state 0x1 ref 1 00000100:00100000:0.0:1789760765.306053:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775604103904 time 775647085326 state 0x0 ref 1 00000100:00100000:0.0:1789760765.306067:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493793152:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760765.306430:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25886720, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc807 00010000:00010000:0.0:1789760765.306440:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e062fa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.306457:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760765.306462:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760765.306479:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e062fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.306495:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e062fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760765.353037:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973639 is committed 00002000:00010000:0.0:1789760765.353337:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25886720 -> 25890816 00010000:00010000:0.0:1789760765.353410:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e062fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.353424:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e062fa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.353433:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e062fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.353440:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760765.353444:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e062fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.353454:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e062fa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.353472:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e062fa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.353479:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e062fa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.353610:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493793152:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47541us (47780us total) trans 4294973639 rc 0/0 00000100:00100000:0.0:1789760765.353620:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6350 00000100:00100000:0.0:1789760765.353630:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775604103904 time 775647085326 state 0x2 ref 0 00000100:00100000:1.0:1789760765.358368:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760765.358609:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493793280 00000100:00100000:2.0:1789760765.358619:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493793280 00000100:00100000:2.0:1789760765.358677:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775604103904 time 775647085326 state 0x0 ref 1 00000100:00100000:2.0:1789760765.358692:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775647345742 time 775647085326 state 0x1 ref 1 00000100:00100000:2.0:1789760765.358756:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775647345742 time 775699789873 state 0x0 ref 1 00000100:00100000:2.0:1789760765.358770:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493793280:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760765.358831:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25890816, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc807 00010000:00010000:2.0:1789760765.358841:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06301 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.358858:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760765.358862:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.358869:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06301 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.358883:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06301 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.397675:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25890816 -> 25894912 00010000:00010000:3.0:1789760765.397696:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06301 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.397716:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06301 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.397726:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06301 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.397733:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760765.397738:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06301 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.397747:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06301 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.397767:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06301 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.397775:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06301 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.397881:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493793280:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39109us (39529us total) trans 4294973640 rc 0/0 00000100:00100000:3.0:1789760765.397892:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6351 00000100:00100000:3.0:1789760765.397902:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775647345742 time 775699789873 state 0x2 ref 0 00000001:00080000:1.0:1789760765.397993:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973640 is committed 00000100:00100000:1.0:1789760765.402257:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760765.402425:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493793408 00000100:00100000:2.0:1789760765.402436:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493793408 00000100:00100000:2.0:1789760765.402491:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775647345742 time 775699789873 state 0x0 ref 1 00000100:00100000:2.0:1789760765.402507:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775700050289 time 775699789873 state 0x1 ref 1 00000100:00100000:2.0:1789760765.402574:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775700050289 time 775743608784 state 0x0 ref 1 00000100:00100000:2.0:1789760765.402587:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493793408:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760765.402637:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25894912, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc807 00010000:00010000:2.0:1789760765.402646:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06308 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.402660:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760765.402665:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.402671:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06308 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.402685:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06308 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760765.439918:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973641 is committed 00002000:00010000:3.0:1789760765.440162:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25894912 -> 25899008 00010000:00010000:3.0:1789760765.440179:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06308 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.440191:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06308 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.440199:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06308 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.440206:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760765.440211:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06308 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.440220:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06308 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.440239:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06308 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.440247:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06308 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.440376:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493793408:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37785us (38122us total) trans 4294973641 rc 0/0 00000100:00100000:3.0:1789760765.440387:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6352 00000100:00100000:3.0:1789760765.440397:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775700050289 time 775743608784 state 0x2 ref 0 00000100:00100000:0.0:1789760765.449285:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760765.449514:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493793536 00000100:00100000:3.0:1789760765.449581:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493793536 00000100:00100000:2.0:1789760765.449735:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775700050289 time 775743608784 state 0x0 ref 1 00000100:00100000:2.0:1789760765.449751:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775743869200 time 775743608784 state 0x1 ref 1 00000100:00100000:2.0:1789760765.449816:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775743869200 time 775790850859 state 0x0 ref 1 00000100:00100000:2.0:1789760765.449829:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb17c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493793536:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760765.449884:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25899008, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc807 00010000:00010000:2.0:1789760765.449893:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0630f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.449910:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760765.449925:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.449932:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0630f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.449946:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0630f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.498518:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760765.504070:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973642 is committed 00002000:00010000:1.0:1789760765.504184:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25899008 -> 25903104 00010000:00010000:1.0:1789760765.504200:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0630f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.504218:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0630f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.504227:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0630f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.504236:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760765.504240:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0630f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.504249:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0630f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.504268:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0630f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.504276:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0630f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.504718:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb17c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493793536:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54885us (55437us total) trans 4294973642 rc 0/0 00000100:00100000:1.0:1789760765.504728:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6353 00000100:00100000:1.0:1789760765.504737:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775743869200 time 775790850859 state 0x2 ref 0 00000100:00100000:1.0:1789760765.511106:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760765.511219:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493793664 00000100:00100000:0.0:1789760765.511229:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493793664 00000100:00100000:0.0:1789760765.511286:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775743869200 time 775790850859 state 0x0 ref 1 00000100:00100000:0.0:1789760765.511311:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775791111275 time 775790850859 state 0x1 ref 1 00000100:00100000:0.0:1789760765.511389:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775791111275 time 775852424422 state 0x0 ref 1 00000100:00100000:0.0:1789760765.511403:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493793664:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760765.511453:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25903104, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc807 00010000:00010000:0.0:1789760765.511462:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06316 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.511478:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760765.511483:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760765.511489:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06316 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.511505:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06316 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760765.557770:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973643 is committed 00002000:00010000:0.0:1789760765.557851:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25903104 -> 25907200 00010000:00010000:0.0:1789760765.557867:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06316 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.557880:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06316 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.557890:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06316 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.557897:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760765.557902:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06316 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.557922:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06316 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.557944:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06316 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.557952:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06316 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.558070:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493793664:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46664us (46968us total) trans 4294973643 rc 0/0 00000100:00100000:0.0:1789760765.558080:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6354 00000100:00100000:0.0:1789760765.558090:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775791111275 time 775852424422 state 0x2 ref 0 00000100:00100000:0.0:1789760765.564363:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760765.568047:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493793792 00000100:00100000:2.0:1789760765.568063:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493793792 00000100:00100000:3.0:1789760765.568821:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775791111275 time 775852424422 state 0x0 ref 1 00000100:00100000:3.0:1789760765.570786:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775852684838 time 775852424422 state 0x1 ref 1 00000100:00100000:3.0:1789760765.570835:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775852684838 time 775911870255 state 0x0 ref 1 00000100:00100000:3.0:1789760765.570857:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493793792:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760765.571140:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25907200, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc807 00010000:00010000:3.0:1789760765.571151:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0631d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.571167:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760765.571172:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.571179:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0631d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.571194:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0631d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760765.648320:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973644 is committed 00002000:00010000:3.0:1789760765.648892:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25907200 -> 25911296 00010000:00010000:3.0:1789760765.648907:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0631d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.648920:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0631d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.648930:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0631d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.648938:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760765.648943:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0631d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.648952:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0631d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.649001:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0631d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.649010:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0631d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.649103:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493793792:12345-192.168.203.60@tcp:4:dd.0 Request processed in 78253us (84753us total) trans 4294973644 rc 0/0 00000100:00100000:3.0:1789760765.649113:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6355 00000100:00100000:3.0:1789760765.649131:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775852684838 time 775911870255 state 0x2 ref 0 00000100:00100000:3.0:1789760765.653948:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760765.654056:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493793920 00000100:00100000:2.0:1789760765.654066:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493793920 00000100:00100000:2.0:1789760765.654121:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775852684838 time 775911870255 state 0x0 ref 1 00000100:00100000:2.0:1789760765.654136:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775912130671 time 775911870255 state 0x1 ref 1 00000100:00100000:2.0:1789760765.654169:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775912130671 time 775995204809 state 0x0 ref 1 00000100:00100000:2.0:1789760765.654182:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493793920:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760765.654228:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25911296, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xc807 00010000:00010000:2.0:1789760765.654238:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06324 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.654253:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760765.654258:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.654264:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06324 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.654276:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06324 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760765.748448:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973645 is committed 00002000:00010000:3.0:1789760765.748599:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25911296 -> 25915392 00002000:00010000:3.0:1789760765.748603:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760764 -> 1789760765 00002000:00010000:3.0:1789760765.748606:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760764 -> 1789760765 00010000:00010000:3.0:1789760765.748618:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06324 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.748631:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06324 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.748639:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06324 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.748646:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760765.748651:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06324 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.748660:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06324 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.748681:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06324 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.748699:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06324 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.748799:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493793920:12345-192.168.203.60@tcp:4:dd.0 Request processed in 94613us (94856us total) trans 4294973645 rc 0/0 00000100:00100000:3.0:1789760765.748809:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6356 00000100:00100000:3.0:1789760765.748820:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775912130671 time 775995204809 state 0x2 ref 0 00000100:00100000:1.0:1789760765.752827:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760765.753283:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493794048 00000100:00100000:2.0:1789760765.753292:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493794048 00000100:00100000:2.0:1789760765.753368:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775912130671 time 775995204809 state 0x0 ref 1 00000100:00100000:2.0:1789760765.753383:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775995465225 time 775995204809 state 0x1 ref 1 00000100:00100000:2.0:1789760765.753415:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775995465225 time 776094450956 state 0x0 ref 1 00000100:00100000:2.0:1789760765.753428:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493794048:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760765.753478:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25915392, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc807 00010000:00010000:2.0:1789760765.753487:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0632b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.753502:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760765.753506:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.753513:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0632b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.753527:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0632b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760765.815325:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25915392 -> 25919488 00010000:00010000:2.0:1789760765.815344:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0632b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.815393:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0632b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.815404:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0632b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.815411:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760765.815417:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0632b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.815426:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0632b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.815447:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0632b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760765.815455:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973646 is committed 00010000:00010000:2.0:1789760765.815456:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0632b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760765.815565:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493794048:12345-192.168.203.60@tcp:4:dd.0 Request processed in 62134us (62741us total) trans 4294973646 rc 0/0 00000100:00100000:2.0:1789760765.815575:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6357 00000100:00100000:2.0:1789760765.815585:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775995465225 time 776094450956 state 0x2 ref 0 00000100:00100000:0.0:1789760765.828015:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760765.828116:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493794176 00000100:00100000:1.0:1789760765.828126:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493794176 00000100:00100000:1.0:1789760765.828183:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 775995465225 time 776094450956 state 0x0 ref 1 00000100:00100000:1.0:1789760765.828198:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776094711372 time 776094450956 state 0x1 ref 1 00000100:00100000:1.0:1789760765.828231:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776094711372 time 776169265831 state 0x0 ref 1 00000100:00100000:1.0:1789760765.828244:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493794176:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760765.828312:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25919488, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc807 00010000:00010000:1.0:1789760765.828322:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06332 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.828338:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760765.828343:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760765.828357:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06332 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.828373:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06332 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760765.904622:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25919488 -> 25923584 00010000:00010000:1.0:1789760765.904644:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06332 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.904658:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06332 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.904667:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06332 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.905157:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760765.905163:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06332 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.905173:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06332 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.905194:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06332 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.905202:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06332 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.905310:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493794176:12345-192.168.203.60@tcp:4:dd.0 Request processed in 77050us (77286us total) trans 4294973647 rc 0/0 00000100:00100000:1.0:1789760765.905321:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6358 00000100:00100000:1.0:1789760765.905331:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776094711372 time 776169265831 state 0x2 ref 0 00000001:00080000:1.0:1789760765.906623:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973647 is committed 00000100:00100000:3.0:1789760765.912415:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760765.914615:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493794304 00000100:00100000:0.0:1789760765.914628:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493794304 00000100:00100000:0.0:1789760765.914703:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776094711372 time 776169265831 state 0x0 ref 1 00000100:00100000:0.0:1789760765.914721:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776169526247 time 776169265831 state 0x1 ref 1 00000100:00100000:0.0:1789760765.914754:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776169526247 time 776255789390 state 0x0 ref 1 00000100:00100000:0.0:1789760765.914768:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493794304:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760765.914823:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25923584, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc807 00010000:00010000:0.0:1789760765.914832:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06339 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.914848:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:0.0:1789760765.914853:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760765.914859:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06339 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.914878:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06339 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.967637:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25923584 -> 25927680 00010000:00010000:3.0:1789760765.967658:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06339 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.967671:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06339 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.967680:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06339 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.967688:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760765.967693:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06339 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.967702:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06339 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.967722:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06339 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.967730:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06339 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.967890:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493794304:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53119us (55538us total) trans 4294973648 rc 0/0 00000100:00100000:3.0:1789760765.967905:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6359 00000100:00100000:3.0:1789760765.967915:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776169526247 time 776255789390 state 0x2 ref 0 00000001:00080000:2.0:1789760765.969199:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973648 is committed 00000100:00100000:0.0:1789760765.971235:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760765.971307:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493794432 00000100:00100000:3.0:1789760765.971317:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493794432 00000100:00100000:3.0:1789760765.971409:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776169526247 time 776255789390 state 0x0 ref 1 00000100:00100000:3.0:1789760765.971425:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776256049806 time 776255789390 state 0x1 ref 1 00000100:00100000:3.0:1789760765.971456:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776256049806 time 776312491619 state 0x0 ref 1 00000100:00100000:3.0:1789760765.971470:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493794432:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760765.971519:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25927680, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc807 00010000:00010000:3.0:1789760765.971528:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06340 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.971544:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760765.971548:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.971555:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06340 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.971569:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06340 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.998204:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25927680 -> 25931776 00000001:00080000:2.0:1789760765.998205:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973649 is committed 00010000:00010000:3.0:1789760765.998222:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06340 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.998237:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06340 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.998247:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06340 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.998258:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760765.998263:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06340 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.998272:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06340 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.998292:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06340 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.998308:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06340 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.998405:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493794432:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26933us (27172us total) trans 4294973649 rc 0/0 00000100:00100000:3.0:1789760765.998414:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6360 00000100:00100000:3.0:1789760765.998423:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776256049806 time 776312491619 state 0x2 ref 0 00000100:00100000:0.0:1789760766.002514:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760766.002746:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493794560 00000100:00100000:2.0:1789760766.002757:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493794560 00000100:00100000:2.0:1789760766.002819:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776256049806 time 776312491619 state 0x0 ref 1 00000100:00100000:2.0:1789760766.002835:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776312752035 time 776312491619 state 0x1 ref 1 00000100:00100000:2.0:1789760766.002866:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776312752035 time 776343901699 state 0x0 ref 1 00000100:00100000:2.0:1789760766.002885:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493794560:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760766.002939:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25931776, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc807 00010000:00010000:2.0:1789760766.002948:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06347 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.003000:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760766.003006:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.003012:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06347 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.003124:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06347 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.041274:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973650 is committed 00002000:00010000:1.0:1789760766.041460:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25931776 -> 25935872 00010000:00010000:1.0:1789760766.041479:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06347 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.041491:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06347 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.041499:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06347 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.041506:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760766.041511:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06347 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.041520:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06347 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.041538:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06347 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.041545:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06347 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.041661:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493794560:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38774us (39151us total) trans 4294973650 rc 0/0 00000100:00100000:1.0:1789760766.041672:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6361 00000100:00100000:1.0:1789760766.041682:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776312752035 time 776343901699 state 0x2 ref 0 00000100:00100000:0.0:1789760766.049207:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760766.049350:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493794688 00000100:00100000:1.0:1789760766.049359:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493794688 00000100:00100000:1.0:1789760766.049489:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776312752035 time 776343901699 state 0x0 ref 1 00000100:00100000:1.0:1789760766.049506:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776344162115 time 776343901699 state 0x1 ref 1 00000100:00100000:1.0:1789760766.049540:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776344162115 time 776390575387 state 0x0 ref 1 00000100:00100000:1.0:1789760766.049552:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493794688:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760766.049828:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25935872, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc807 00010000:00010000:1.0:1789760766.049839:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0634e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.049865:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760766.049876:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.049884:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0634e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.049898:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0634e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760766.085750:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973651 is committed 00002000:00010000:1.0:1789760766.085773:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25935872 -> 25939968 00010000:00010000:1.0:1789760766.085789:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0634e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.085803:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0634e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.085813:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0634e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.085820:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760766.085825:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0634e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.085834:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0634e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.085851:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0634e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.085858:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0634e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.085945:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493794688:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36389us (36745us total) trans 4294973651 rc 0/0 00000100:00100000:1.0:1789760766.085955:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6362 00000100:00100000:1.0:1789760766.085989:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776344162115 time 776390575387 state 0x2 ref 0 00000100:00100000:2.0:1789760766.089454:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760766.089809:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493794816 00000100:00100000:3.0:1789760766.089819:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493794816 00000100:00100000:3.0:1789760766.089875:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776344162115 time 776390575387 state 0x0 ref 1 00000100:00100000:3.0:1789760766.089890:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776390835803 time 776390575387 state 0x1 ref 1 00000100:00100000:3.0:1789760766.089957:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776390835803 time 776430991936 state 0x0 ref 1 00000100:00100000:3.0:1789760766.089997:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493794816:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760766.090067:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25939968, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc807 00010000:00010000:3.0:1789760766.090080:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06355 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.090094:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760766.090099:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.090105:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06355 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.090118:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06355 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760766.115117:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25939968 -> 25944064 00010000:00010000:3.0:1789760766.115136:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06355 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760766.115150:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973652 is committed 00010000:00010000:3.0:1789760766.115150:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06355 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.115169:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06355 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.115176:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760766.115181:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06355 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.115190:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06355 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.115208:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06355 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.115215:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06355 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.115314:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493794816:12345-192.168.203.60@tcp:4:dd.0 Request processed in 25315us (25864us total) trans 4294973652 rc 0/0 00000100:00100000:3.0:1789760766.115324:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6363 00000100:00100000:3.0:1789760766.115334:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776390835803 time 776430991936 state 0x2 ref 0 00000100:00100000:0.0:1789760766.120940:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760766.121015:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493794944 00000100:00100000:3.0:1789760766.121035:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493794944 00000100:00100000:3.0:1789760766.121088:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776390835803 time 776430991936 state 0x0 ref 1 00000100:00100000:3.0:1789760766.121111:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776431252352 time 776430991936 state 0x1 ref 1 00000100:00100000:3.0:1789760766.121133:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776431252352 time 776462169077 state 0x0 ref 1 00000100:00100000:3.0:1789760766.121145:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493794944:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760766.121189:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25944064, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc807 00010000:00010000:3.0:1789760766.121197:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0635c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.121211:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760766.121215:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.121221:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0635c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.121235:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0635c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760766.187261:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25944064 -> 25948160 00010000:00010000:3.0:1789760766.187281:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0635c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.187303:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0635c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.187312:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0635c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.187319:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760766.187325:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0635c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.187334:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0635c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.187352:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0635c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.187359:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0635c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.187448:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493794944:12345-192.168.203.60@tcp:4:dd.0 Request processed in 66300us (66512us total) trans 4294973653 rc 0/0 00000100:00100000:3.0:1789760766.187458:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6364 00000100:00100000:3.0:1789760766.187468:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776431252352 time 776462169077 state 0x2 ref 0 00000001:00080000:2.0:1789760766.187665:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973653 is committed 00000100:00100000:1.0:1789760766.192778:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760766.192873:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493795072 00000100:00100000:2.0:1789760766.192883:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493795072 00000100:00100000:2.0:1789760766.192938:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776431252352 time 776462169077 state 0x0 ref 1 00000100:00100000:2.0:1789760766.192993:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776462429493 time 776462169077 state 0x1 ref 1 00000100:00100000:2.0:1789760766.193019:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776462429493 time 776534054817 state 0x0 ref 1 00000100:00100000:2.0:1789760766.193110:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493795072:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760766.193156:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25948160, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc807 00010000:00010000:2.0:1789760766.193167:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06363 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.193182:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760766.193186:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.193192:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06363 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.193206:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06363 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760766.225621:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973654 is committed 00002000:00010000:2.0:1789760766.227212:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25948160 -> 25952256 00010000:00010000:2.0:1789760766.227229:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06363 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.227241:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06363 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.227249:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06363 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.227255:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760766.227259:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06363 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.227266:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06363 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.227289:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06363 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.227307:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06363 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.227406:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493795072:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34294us (34633us total) trans 4294973654 rc 0/0 00000100:00100000:2.0:1789760766.227417:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6365 00000100:00100000:2.0:1789760766.227427:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776462429493 time 776534054817 state 0x2 ref 0 00000100:00100000:1.0:1789760766.231582:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760766.232038:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493795200 00000100:00100000:2.0:1789760766.232048:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493795200 00000100:00100000:2.0:1789760766.232103:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776462429493 time 776534054817 state 0x0 ref 1 00000100:00100000:2.0:1789760766.232117:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776534315233 time 776534054817 state 0x1 ref 1 00000100:00100000:2.0:1789760766.232149:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776534315233 time 776573184603 state 0x0 ref 1 00000100:00100000:2.0:1789760766.232164:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493795200:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760766.232215:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25952256, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc807 00010000:00010000:2.0:1789760766.232225:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0636a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.232241:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760766.232247:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.232254:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0636a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.232267:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0636a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760766.257998:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25952256 -> 25956352 00010000:00010000:2.0:1789760766.258017:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0636a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.258029:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0636a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.258037:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0636a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.258043:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760766.258047:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0636a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.258055:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0636a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.258071:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0636a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.258078:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0636a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.258163:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493795200:12345-192.168.203.60@tcp:4:dd.0 Request processed in 25999us (26586us total) trans 4294973655 rc 0/0 00000100:00100000:2.0:1789760766.258184:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6366 00000100:00100000:2.0:1789760766.258194:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776534315233 time 776573184603 state 0x2 ref 0 00000001:00080000:1.0:1789760766.259106:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973655 is committed 00000100:00100000:0.0:1789760766.265680:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760766.265878:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493795328 00000100:00100000:3.0:1789760766.265889:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493795328 00000100:00100000:3.0:1789760766.266730:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776534315233 time 776573184603 state 0x0 ref 1 00000100:00100000:3.0:1789760766.266748:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776573445019 time 776573184603 state 0x1 ref 1 00000100:00100000:3.0:1789760766.266793:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776573445019 time 776607828202 state 0x0 ref 1 00000100:00100000:3.0:1789760766.266808:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493795328:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760766.266858:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25956352, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc807 00010000:00010000:3.0:1789760766.266867:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06371 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.266883:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760766.266887:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.266893:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06371 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.266906:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06371 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760766.320688:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973656 is committed 00002000:00010000:0.0:1789760766.320802:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25956352 -> 25960448 00010000:00010000:0.0:1789760766.320818:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06371 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.320831:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06371 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.320840:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06371 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.320847:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760766.320853:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06371 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.320863:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06371 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.320888:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06371 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.320896:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06371 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.321028:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493795328:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54219us (55353us total) trans 4294973656 rc 0/0 00000100:00100000:0.0:1789760766.321038:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6367 00000100:00100000:0.0:1789760766.321049:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776573445019 time 776607828202 state 0x2 ref 0 00000100:00100000:0.0:1789760766.326559:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760766.326730:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493795456 00000100:00100000:3.0:1789760766.326740:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493795456 00000100:00100000:3.0:1789760766.326792:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776573445019 time 776607828202 state 0x0 ref 1 00000100:00100000:3.0:1789760766.326808:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776608088618 time 776607828202 state 0x1 ref 1 00000100:00100000:3.0:1789760766.326893:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776608088618 time 776667928635 state 0x0 ref 1 00000100:00100000:3.0:1789760766.326905:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493795456:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760766.326956:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25960448, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc807 00010000:00010000:3.0:1789760766.327003:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06378 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.327018:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760766.327023:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.327029:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06378 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.327044:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06378 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.354767:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973657 is committed 00002000:00010000:2.0:1789760766.355578:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25960448 -> 25964544 00010000:00010000:2.0:1789760766.355605:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06378 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.355618:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06378 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.355627:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06378 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.355635:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760766.355640:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06378 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.355649:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06378 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.355669:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06378 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.355682:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06378 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.355783:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493795456:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28874us (29228us total) trans 4294973657 rc 0/0 00000100:00100000:2.0:1789760766.355793:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6368 00000100:00100000:2.0:1789760766.355803:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776608088618 time 776667928635 state 0x2 ref 0 00000100:00100000:2.0:1789760766.368136:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760766.368215:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493795584 00000100:00100000:1.0:1789760766.368225:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493795584 00000100:00100000:1.0:1789760766.368280:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776608088618 time 776667928635 state 0x0 ref 1 00000100:00100000:1.0:1789760766.368305:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776668189051 time 776667928635 state 0x1 ref 1 00000100:00100000:1.0:1789760766.368386:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776668189051 time 776709419852 state 0x0 ref 1 00000100:00100000:1.0:1789760766.368401:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493795584:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760766.368451:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25964544, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc807 00010000:00010000:1.0:1789760766.368460:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0637f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.368475:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760766.368479:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.368485:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0637f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.368501:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0637f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760766.408851:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25964544 -> 25968640 00010000:00010000:2.0:1789760766.408873:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0637f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.408887:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0637f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.408896:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0637f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.408905:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760766.408909:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0637f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.408919:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0637f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.408956:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0637f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.408984:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0637f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.409088:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493795584:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40685us (40955us total) trans 4294973658 rc 0/0 00000100:00100000:2.0:1789760766.409099:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6369 00000100:00100000:2.0:1789760766.409110:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776668189051 time 776709419852 state 0x2 ref 0 00000001:00080000:0.0:1789760766.410503:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973658 is committed 00000100:00100000:0.0:1789760766.417572:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760766.418370:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493795712 00000100:00100000:2.0:1789760766.418379:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493795712 00000100:00100000:2.0:1789760766.418433:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776668189051 time 776709419852 state 0x0 ref 1 00000100:00100000:2.0:1789760766.418448:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776709680268 time 776709419852 state 0x1 ref 1 00000100:00100000:2.0:1789760766.418479:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776709680268 time 776759515303 state 0x0 ref 1 00000100:00100000:2.0:1789760766.418491:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493795712:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760766.418537:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25968640, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc807 00010000:00010000:2.0:1789760766.418546:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06386 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.418562:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760766.418567:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.418574:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06386 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.418587:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06386 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760766.457251:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973659 is committed 00002000:00010000:2.0:1789760766.457433:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25968640 -> 25972736 00010000:00010000:2.0:1789760766.457448:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06386 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.457462:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06386 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.457471:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06386 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.457478:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760766.457483:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06386 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.457492:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06386 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.457512:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06386 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.457519:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06386 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.457714:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493795712:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39219us (40146us total) trans 4294973659 rc 0/0 00000100:00100000:2.0:1789760766.457725:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6370 00000100:00100000:2.0:1789760766.457736:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776709680268 time 776759515303 state 0x2 ref 0 00000100:00100000:1.0:1789760766.464958:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760766.466428:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493795840 00000100:00100000:3.0:1789760766.466443:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493795840 00000100:00100000:3.0:1789760766.466508:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776709680268 time 776759515303 state 0x0 ref 1 00000100:00100000:3.0:1789760766.466525:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776759775719 time 776759515303 state 0x1 ref 1 00000100:00100000:3.0:1789760766.466589:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776759775719 time 776807624661 state 0x0 ref 1 00000100:00100000:3.0:1789760766.467114:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493795840:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760766.467167:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25972736, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc807 00010000:00010000:3.0:1789760766.467176:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0638d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.467193:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760766.467197:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.467205:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0638d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.467220:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0638d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760766.501699:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973660 is committed 00002000:00010000:1.0:1789760766.501701:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25972736 -> 25976832 00010000:00010000:1.0:1789760766.501717:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0638d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.501731:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0638d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.501741:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0638d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.501748:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760766.501753:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0638d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.501763:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0638d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.501783:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0638d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.501802:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0638d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.502264:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493795840:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35290us (36943us total) trans 4294973660 rc 0/0 00000100:00100000:1.0:1789760766.502278:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6371 00000100:00100000:1.0:1789760766.502288:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776759775719 time 776807624661 state 0x2 ref 0 00000100:00100000:0.0:1789760766.506342:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760766.506423:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493795968 00000100:00100000:3.0:1789760766.506433:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493795968 00000100:00100000:3.0:1789760766.506492:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776759775719 time 776807624661 state 0x0 ref 1 00000100:00100000:3.0:1789760766.506508:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776807885077 time 776807624661 state 0x1 ref 1 00000100:00100000:3.0:1789760766.506541:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776807885077 time 776847576747 state 0x0 ref 1 00000100:00100000:3.0:1789760766.506555:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493795968:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760766.506607:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25976832, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc807 00010000:00010000:3.0:1789760766.506616:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06394 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.506633:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760766.506637:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.506643:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06394 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.506658:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06394 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.524127:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760766.550985:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973661 is committed 00002000:00010000:3.0:1789760766.551535:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25976832 -> 25980928 00010000:00010000:3.0:1789760766.551552:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06394 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.551570:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06394 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.551578:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06394 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.551584:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760766.551589:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06394 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.551598:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06394 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.551620:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06394 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.551627:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06394 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.552011:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493795968:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45401us (45621us total) trans 4294973661 rc 0/0 00000100:00100000:3.0:1789760766.552133:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6372 00000100:00100000:3.0:1789760766.552143:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776807885077 time 776847576747 state 0x2 ref 0 00000100:00100000:0.0:1789760766.556435:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760766.556508:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493796096 00000100:00100000:2.0:1789760766.556518:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493796096 00000100:00100000:2.0:1789760766.556573:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776807885077 time 776847576747 state 0x0 ref 1 00000100:00100000:2.0:1789760766.556588:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776847837163 time 776847576747 state 0x1 ref 1 00000100:00100000:2.0:1789760766.556617:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776847837163 time 776897652695 state 0x0 ref 1 00000100:00100000:2.0:1789760766.556629:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493796096:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760766.556848:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25980928, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc807 00010000:00010000:2.0:1789760766.556857:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0639b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.556872:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760766.556877:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.556883:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0639b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.556896:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0639b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760766.597495:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973662 is committed 00002000:00010000:2.0:1789760766.597632:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25980928 -> 25985024 00010000:00010000:2.0:1789760766.597649:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0639b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.597661:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0639b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.597670:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0639b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.597677:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760766.597682:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0639b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.597689:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0639b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.597707:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0639b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.597713:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0639b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.597811:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493796096:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41174us (41375us total) trans 4294973662 rc 0/0 00000100:00100000:2.0:1789760766.597835:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6373 00000100:00100000:2.0:1789760766.597845:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776847837163 time 776897652695 state 0x2 ref 0 00000100:00100000:1.0:1789760766.604424:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760766.604542:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493796224 00000100:00100000:2.0:1789760766.604553:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493796224 00000100:00100000:2.0:1789760766.604609:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776847837163 time 776897652695 state 0x0 ref 1 00000100:00100000:2.0:1789760766.604626:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776897913111 time 776897652695 state 0x1 ref 1 00000100:00100000:2.0:1789760766.604656:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776897913111 time 776945691418 state 0x0 ref 1 00000100:00100000:2.0:1789760766.604669:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc14e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493796224:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760766.604720:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25985024, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xc807 00010000:00010000:2.0:1789760766.604729:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e063a2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.604744:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760766.604749:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.604754:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e063a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.604768:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e063a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.659020:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973663 is committed 00002000:00010000:0.0:1789760766.659184:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25985024 -> 25989120 00002000:00010000:0.0:1789760766.659189:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760765 -> 1789760766 00002000:00010000:0.0:1789760766.659192:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760765 -> 1789760766 00010000:00010000:0.0:1789760766.659204:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e063a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.659217:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e063a2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.659226:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e063a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.659234:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760766.659238:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e063a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.659247:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e063a2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.659266:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e063a2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.659274:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e063a2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.659482:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc14e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493796224:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54808us (55060us total) trans 4294973663 rc 0/0 00000100:00100000:0.0:1789760766.659493:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6374 00000100:00100000:0.0:1789760766.659503:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776897913111 time 776945691418 state 0x2 ref 0 00000100:00100000:0.0:1789760766.666108:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760766.666207:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493796352 00000100:00100000:1.0:1789760766.666218:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493796352 00000100:00100000:1.0:1789760766.666274:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776897913111 time 776945691418 state 0x0 ref 1 00000100:00100000:1.0:1789760766.666304:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776945951834 time 776945691418 state 0x1 ref 1 00000100:00100000:1.0:1789760766.666341:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776945951834 time 777007375783 state 0x0 ref 1 00000100:00100000:1.0:1789760766.666463:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493796352:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760766.666517:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25989120, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc807 00010000:00010000:1.0:1789760766.666526:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e063a9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.666541:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760766.666546:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.666552:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e063a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.666569:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e063a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760766.720478:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973664 is committed 00002000:00010000:0.0:1789760766.720641:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25989120 -> 25993216 00010000:00010000:0.0:1789760766.720657:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e063a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.720675:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e063a9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.720684:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e063a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.720691:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760766.720695:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e063a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.720704:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e063a9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.720723:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e063a9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.720730:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e063a9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.720831:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493796352:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54366us (54727us total) trans 4294973664 rc 0/0 00000100:00100000:0.0:1789760766.720841:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6375 00000100:00100000:0.0:1789760766.720853:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776945951834 time 777007375783 state 0x2 ref 0 00000100:00100000:2.0:1789760766.724009:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760766.724094:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493796480 00000100:00100000:3.0:1789760766.724104:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493796480 00000100:00100000:3.0:1789760766.724176:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 776945951834 time 777007375783 state 0x0 ref 1 00000100:00100000:3.0:1789760766.724190:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777007636199 time 777007375783 state 0x1 ref 1 00000100:00100000:3.0:1789760766.724214:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777007636199 time 777065249875 state 0x0 ref 1 00000100:00100000:3.0:1789760766.724227:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493796480:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760766.724277:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25993216, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc807 00010000:00010000:3.0:1789760766.724286:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e063b0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.724312:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760766.724317:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.724323:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e063b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.724336:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e063b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760766.774121:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973665 is committed 00002000:00010000:1.0:1789760766.774280:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25993216 -> 25997312 00010000:00010000:1.0:1789760766.774304:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e063b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.774316:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e063b0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.774331:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e063b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.774340:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760766.774353:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e063b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.774362:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e063b0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.774381:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e063b0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.774389:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e063b0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.774479:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493796480:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50249us (50514us total) trans 4294973665 rc 0/0 00000100:00100000:1.0:1789760766.774490:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6376 00000100:00100000:1.0:1789760766.774501:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777007636199 time 777065249875 state 0x2 ref 0 00000100:00100000:1.0:1789760766.781465:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760766.781613:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493796608 00000100:00100000:2.0:1789760766.781623:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493796608 00000100:00100000:2.0:1789760766.781681:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777007636199 time 777065249875 state 0x0 ref 1 00000100:00100000:2.0:1789760766.781696:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777065510291 time 777065249875 state 0x1 ref 1 00000100:00100000:2.0:1789760766.781726:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777065510291 time 777122760979 state 0x0 ref 1 00000100:00100000:2.0:1789760766.781739:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493796608:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760766.781787:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25997312, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc807 00010000:00010000:2.0:1789760766.781795:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e063b7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.781811:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:2.0:1789760766.781816:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.781822:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e063b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.781836:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e063b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760766.810812:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973666 is committed 00002000:00010000:1.0:1789760766.810954:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 25997312 -> 26001408 00010000:00010000:1.0:1789760766.811051:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e063b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.811072:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e063b7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.811081:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e063b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.811088:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760766.811093:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e063b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.811103:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e063b7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.811124:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e063b7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.811131:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e063b7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.811337:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493796608:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29595us (29876us total) trans 4294973666 rc 0/0 00000100:00100000:0.0:1789760766.811511:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6377 00000100:00100000:0.0:1789760766.811522:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777065510291 time 777122760979 state 0x2 ref 0 00000100:00100000:0.0:1789760766.817105:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760766.821030:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493796736 00000100:00100000:3.0:1789760766.821045:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493796736 00000100:00100000:3.0:1789760766.821108:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777065510291 time 777122760979 state 0x0 ref 1 00000100:00100000:3.0:1789760766.821123:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777123021395 time 777122760979 state 0x1 ref 1 00000100:00100000:3.0:1789760766.821149:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777123021395 time 777162185482 state 0x0 ref 1 00000100:00100000:3.0:1789760766.821162:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493796736:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760766.821210:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26001408, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc807 00010000:00010000:3.0:1789760766.821219:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e063be lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.821236:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760766.821240:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.821246:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e063be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.821259:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e063be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760766.859110:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26001408 -> 26005504 00010000:00010000:3.0:1789760766.859128:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e063be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.859143:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e063be lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.859152:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e063be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.859159:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760766.859164:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e063be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.859174:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e063be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.859191:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e063be lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.859198:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e063be lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.859309:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493796736:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38143us (42209us total) trans 4294973667 rc 0/0 00000100:00100000:3.0:1789760766.859319:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6378 00000100:00100000:3.0:1789760766.859329:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777123021395 time 777162185482 state 0x2 ref 0 00000001:00080000:1.0:1789760766.861004:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973667 is committed 00000100:00100000:2.0:1789760766.863566:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760766.863702:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493796864 00000100:00100000:0.0:1789760766.863710:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493796864 00000100:00100000:0.0:1789760766.863759:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777123021395 time 777162185482 state 0x0 ref 1 00000100:00100000:0.0:1789760766.863771:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777162445898 time 777162185482 state 0x1 ref 1 00000100:00100000:0.0:1789760766.863846:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777162445898 time 777204881538 state 0x0 ref 1 00000100:00100000:0.0:1789760766.863859:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493796864:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760766.863914:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26005504, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc807 00010000:00010000:0.0:1789760766.863924:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e063c5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.863938:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:0.0:1789760766.863943:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760766.863950:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e063c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.865462:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e063c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760766.903468:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973668 is committed 00002000:00010000:1.0:1789760766.905582:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26005504 -> 26009600 00010000:00010000:1.0:1789760766.905604:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e063c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.905618:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e063c5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.905627:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e063c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.905634:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760766.905639:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e063c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.905648:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e063c5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.905669:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e063c5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.905677:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e063c5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.905932:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493796864:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42070us (42563us total) trans 4294973668 rc 0/0 00000100:00100000:1.0:1789760766.905943:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6379 00000100:00100000:1.0:1789760766.905953:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777162445898 time 777204881538 state 0x2 ref 0 00000100:00100000:2.0:1789760766.917087:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760766.917244:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493796992 00000100:00100000:1.0:1789760766.917256:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493796992 00000100:00100000:1.0:1789760766.917326:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777162445898 time 777204881538 state 0x0 ref 1 00000100:00100000:1.0:1789760766.917342:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777205141954 time 777204881538 state 0x1 ref 1 00000100:00100000:1.0:1789760766.917374:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777205141954 time 777258408981 state 0x0 ref 1 00000100:00100000:1.0:1789760766.917388:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493796992:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760766.917442:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26009600, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc807 00010000:00010000:1.0:1789760766.917452:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e063cc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.917468:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760766.917472:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.917478:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e063cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.917492:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e063cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760766.975308:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973669 is committed 00002000:00010000:1.0:1789760766.975833:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26009600 -> 26013696 00010000:00010000:1.0:1789760766.975848:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e063cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.975860:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e063cc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.975869:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e063cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.975876:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760766.975881:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e063cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.975889:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e063cc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.975908:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e063cc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.975916:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e063cc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.976135:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493796992:12345-192.168.203.60@tcp:4:dd.0 Request processed in 58745us (59053us total) trans 4294973669 rc 0/0 00000100:00100000:1.0:1789760766.976145:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6380 00000100:00100000:1.0:1789760766.976155:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777205141954 time 777258408981 state 0x2 ref 0 00000100:00100000:0.0:1789760766.979881:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760766.980102:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493797120 00000100:00100000:3.0:1789760766.980112:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493797120 00000100:00100000:3.0:1789760766.980171:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777205141954 time 777258408981 state 0x0 ref 1 00000100:00100000:3.0:1789760766.980185:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777258669397 time 777258408981 state 0x1 ref 1 00000100:00100000:3.0:1789760766.980550:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777258669397 time 777321583854 state 0x0 ref 1 00000100:00100000:3.0:1789760766.980564:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493797120:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760766.980614:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26013696, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc807 00010000:00010000:3.0:1789760766.980623:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e063d3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.980639:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760766.980644:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.980649:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e063d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.980663:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e063d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760767.022695:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973670 is committed 00002000:00010000:0.0:1789760767.022808:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26013696 -> 26017792 00010000:00010000:0.0:1789760767.022823:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e063d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.022836:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e063d3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.022845:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e063d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.022851:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760767.022856:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e063d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.022866:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e063d3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.022884:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e063d3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.022891:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e063d3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.023040:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493797120:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42474us (43163us total) trans 4294973670 rc 0/0 00000100:00100000:0.0:1789760767.023051:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6381 00000100:00100000:0.0:1789760767.023061:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777258669397 time 777321583854 state 0x2 ref 0 00000100:00100000:2.0:1789760767.030472:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760767.030633:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493797248 00000100:00100000:3.0:1789760767.030643:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493797248 00000100:00100000:3.0:1789760767.030818:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777258669397 time 777321583854 state 0x0 ref 1 00000100:00100000:3.0:1789760767.030833:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777321844270 time 777321583854 state 0x1 ref 1 00000100:00100000:3.0:1789760767.030865:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777321844270 time 777371900182 state 0x0 ref 1 00000100:00100000:3.0:1789760767.030879:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493797248:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760767.030930:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26017792, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc807 00010000:00010000:3.0:1789760767.030938:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e063da lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.030954:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760767.031106:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.031115:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e063da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.031136:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e063da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760767.031526:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760767.065907:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973671 is committed 00002000:00010000:2.0:1789760767.066272:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26017792 -> 26021888 00010000:00010000:2.0:1789760767.066298:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e063da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.066313:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e063da lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.066323:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e063da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.066331:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760767.066336:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e063da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.066356:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e063da lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.066377:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e063da lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.066384:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e063da lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.066515:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493797248:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35635us (36052us total) trans 4294973671 rc 0/0 00000100:00100000:2.0:1789760767.066527:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6382 00000100:00100000:2.0:1789760767.066537:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777321844270 time 777371900182 state 0x2 ref 0 00000100:00100000:2.0:1789760767.070389:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760767.070488:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493797376 00000100:00100000:1.0:1789760767.070498:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493797376 00000100:00100000:1.0:1789760767.070554:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777321844270 time 777371900182 state 0x0 ref 1 00000100:00100000:1.0:1789760767.070570:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777372160598 time 777371900182 state 0x1 ref 1 00000100:00100000:1.0:1789760767.070605:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777372160598 time 777411640392 state 0x0 ref 1 00000100:00100000:1.0:1789760767.070618:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493797376:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760767.070669:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26021888, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc807 00010000:00010000:1.0:1789760767.070678:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e063e1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.070695:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760767.070700:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760767.070706:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e063e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.070719:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e063e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760767.112986:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973672 is committed 00002000:00010000:0.0:1789760767.113196:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26021888 -> 26025984 00010000:00010000:0.0:1789760767.113211:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e063e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.113223:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e063e1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.113231:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e063e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.113239:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760767.113244:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e063e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.113253:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e063e1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.113272:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e063e1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.113463:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e063e1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.113592:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493797376:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42971us (43210us total) trans 4294973672 rc 0/0 00000100:00100000:0.0:1789760767.113606:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6383 00000100:00100000:0.0:1789760767.113615:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777372160598 time 777411640392 state 0x2 ref 0 00000100:00100000:1.0:1789760767.117190:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760767.118659:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493797504 00000100:00100000:3.0:1789760767.118671:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493797504 00000100:00100000:3.0:1789760767.119048:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777372160598 time 777411640392 state 0x0 ref 1 00000100:00100000:3.0:1789760767.119064:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777411900808 time 777411640392 state 0x1 ref 1 00000100:00100000:3.0:1789760767.119101:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777411900808 time 777460136239 state 0x0 ref 1 00000100:00100000:3.0:1789760767.119114:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493797504:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760767.119167:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26025984, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc807 00010000:00010000:3.0:1789760767.119176:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e063e8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.119192:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760767.119196:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.119203:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e063e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.119217:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e063e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.168886:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973673 is committed 00002000:00010000:0.0:1789760767.169218:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26025984 -> 26030080 00010000:00010000:0.0:1789760767.169234:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e063e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.169247:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e063e8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.169255:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e063e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.169262:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760767.169267:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e063e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.169276:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e063e8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.169304:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e063e8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.169312:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e063e8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.169396:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493797504:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50279us (52212us total) trans 4294973673 rc 0/0 00000100:00100000:0.0:1789760767.169406:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6384 00000100:00100000:0.0:1789760767.169415:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777411900808 time 777460136239 state 0x2 ref 0 00000100:00100000:2.0:1789760767.177759:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760767.179164:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493797632 00000100:00100000:0.0:1789760767.179176:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493797632 00000100:00100000:0.0:1789760767.179236:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777411900808 time 777460136239 state 0x0 ref 1 00000100:00100000:0.0:1789760767.179250:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777460396655 time 777460136239 state 0x1 ref 1 00000100:00100000:0.0:1789760767.179372:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777460396655 time 777520406234 state 0x0 ref 1 00000100:00100000:0.0:1789760767.179385:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493797632:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760767.179434:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26030080, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc807 00010000:00010000:0.0:1789760767.179443:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e063ef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.179460:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:0.0:1789760767.179465:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.179472:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e063ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.179485:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e063ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.260373:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973674 is committed 00002000:00010000:0.0:1789760767.260590:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26030080 -> 26034176 00010000:00010000:0.0:1789760767.260611:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e063ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.260627:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e063ef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.260638:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e063ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.260647:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760767.260654:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e063ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.260665:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e063ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.260693:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e063ef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.260702:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e063ef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.260840:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493797632:12345-192.168.203.60@tcp:4:dd.0 Request processed in 81450us (83084us total) trans 4294973674 rc 0/0 00000100:00100000:0.0:1789760767.260857:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6385 00000100:00100000:0.0:1789760767.260871:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777460396655 time 777520406234 state 0x2 ref 0 00000100:00100000:1.0:1789760767.266193:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760767.266789:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493797760 00000100:00100000:2.0:1789760767.266800:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493797760 00000100:00100000:2.0:1789760767.266860:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777460396655 time 777520406234 state 0x0 ref 1 00000100:00100000:2.0:1789760767.266876:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777520666650 time 777520406234 state 0x1 ref 1 00000100:00100000:2.0:1789760767.266907:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777520666650 time 777607941656 state 0x0 ref 1 00000100:00100000:2.0:1789760767.266922:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493797760:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760767.267027:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26034176, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc807 00010000:00010000:2.0:1789760767.267037:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e063f6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.267052:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:2.0:1789760767.267056:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.267062:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e063f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.267076:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e063f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760767.337830:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973675 is committed 00002000:00010000:0.0:1789760767.337957:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26034176 -> 26038272 00010000:00010000:0.0:1789760767.338003:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e063f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.338018:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e063f6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.338030:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e063f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.338039:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760767.338045:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e063f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.338055:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e063f6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.338079:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e063f6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.338087:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e063f6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.338277:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493797760:12345-192.168.203.60@tcp:4:dd.0 Request processed in 71349us (72085us total) trans 4294973675 rc 0/0 00000100:00100000:3.0:1789760767.338300:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6386 00000100:00100000:3.0:1789760767.338313:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777520666650 time 777607941656 state 0x2 ref 0 00000100:00100000:3.0:1789760767.342916:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760767.343031:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493797888 00000100:00100000:1.0:1789760767.343049:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493797888 00000100:00100000:1.0:1789760767.343143:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777520666650 time 777607941656 state 0x0 ref 1 00000100:00100000:1.0:1789760767.343165:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777608202072 time 777607941656 state 0x1 ref 1 00000100:00100000:1.0:1789760767.343206:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777608202072 time 777684240426 state 0x0 ref 1 00000100:00100000:1.0:1789760767.343221:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8555880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493797888:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760767.343301:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26038272, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc807 00010000:00010000:1.0:1789760767.343313:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e063fd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.343346:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760767.343351:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760767.343358:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e063fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.343376:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e063fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760767.402192:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973676 is committed 00002000:00010000:2.0:1789760767.402326:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26038272 -> 26042368 00010000:00010000:2.0:1789760767.402356:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e063fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.402371:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e063fd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.402381:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e063fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.402389:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760767.402394:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e063fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.402405:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e063fd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.402428:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e063fd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.402435:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e063fd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.402555:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8555880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493797888:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59330us (59643us total) trans 4294973676 rc 0/0 00000100:00100000:2.0:1789760767.402567:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6387 00000100:00100000:2.0:1789760767.402580:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777608202072 time 777684240426 state 0x2 ref 0 00000100:00100000:2.0:1789760767.406158:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760767.406269:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493798016 00000100:00100000:0.0:1789760767.406281:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493798016 00000100:00100000:0.0:1789760767.406432:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777608202072 time 777684240426 state 0x0 ref 1 00000100:00100000:0.0:1789760767.406454:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777684500842 time 777684240426 state 0x1 ref 1 00000100:00100000:0.0:1789760767.406531:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777684500842 time 777747565069 state 0x0 ref 1 00000100:00100000:0.0:1789760767.406548:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493798016:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760767.406619:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26042368, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc807 00010000:00010000:0.0:1789760767.406632:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06404 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.406653:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:0.0:1789760767.406659:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.406666:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06404 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.406693:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06404 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760767.448495:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973677 is committed 00002000:00010000:1.0:1789760767.448623:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26042368 -> 26046464 00010000:00010000:1.0:1789760767.448642:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06404 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.448658:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06404 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.448667:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06404 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.448676:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760767.448681:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06404 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.448691:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06404 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.448715:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06404 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.448725:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06404 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.448913:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493798016:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42361us (42762us total) trans 4294973677 rc 0/0 00000100:00100000:1.0:1789760767.448927:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6388 00000100:00100000:1.0:1789760767.448940:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777684500842 time 777747565069 state 0x2 ref 0 00000100:00100000:0.0:1789760767.453432:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760767.453578:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493798144 00000100:00100000:3.0:1789760767.453592:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493798144 00000100:00100000:3.0:1789760767.453669:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777684500842 time 777747565069 state 0x0 ref 1 00000100:00100000:3.0:1789760767.453689:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777747825485 time 777747565069 state 0x1 ref 1 00000100:00100000:3.0:1789760767.453725:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777747825485 time 777794760302 state 0x0 ref 1 00000100:00100000:3.0:1789760767.453740:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493798144:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760767.453816:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26046464, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc807 00010000:00010000:3.0:1789760767.453828:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0640b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.453850:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760767.453855:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.453864:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0640b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.453881:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0640b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760767.524775:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973678 is committed 00002000:00010000:0.0:1789760767.524907:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26046464 -> 26050560 00010000:00010000:0.0:1789760767.524928:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0640b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.524946:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0640b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.524957:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0640b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.525037:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760767.525043:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0640b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.525053:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0640b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.525079:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0640b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.525087:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0640b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.525214:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493798144:12345-192.168.203.60@tcp:4:dd.0 Request processed in 71469us (71793us total) trans 4294973678 rc 0/0 00000100:00100000:0.0:1789760767.525227:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6389 00000100:00100000:0.0:1789760767.525240:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777747825485 time 777794760302 state 0x2 ref 0 00000100:00100000:2.0:1789760767.532505:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760767.532639:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493798272 00000100:00100000:3.0:1789760767.532653:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493798272 00000100:00100000:3.0:1789760767.532733:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777747825485 time 777794760302 state 0x0 ref 1 00000100:00100000:3.0:1789760767.532755:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777795020718 time 777794760302 state 0x1 ref 1 00000100:00100000:3.0:1789760767.532882:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777795020718 time 777873916855 state 0x0 ref 1 00000100:00100000:3.0:1789760767.532900:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493798272:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760767.533079:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26050560, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc807 00010000:00010000:3.0:1789760767.533091:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06412 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.533111:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760767.533117:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.533125:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06412 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.533141:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06412 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.549989:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760767.580486:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973679 is committed 00002000:00010000:2.0:1789760767.580529:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26050560 -> 26054656 00010000:00010000:2.0:1789760767.580553:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06412 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.580569:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06412 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.580580:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06412 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.580588:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760767.580593:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06412 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.580604:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06412 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.580627:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06412 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.580635:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06412 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.580755:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493798272:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47851us (48257us total) trans 4294973679 rc 0/0 00000100:00100000:2.0:1789760767.580768:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6390 00000100:00100000:2.0:1789760767.580781:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777795020718 time 777873916855 state 0x2 ref 0 00000100:00100000:3.0:1789760767.589526:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760767.589640:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493798400 00000100:00100000:2.0:1789760767.589654:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493798400 00000100:00100000:2.0:1789760767.589732:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777795020718 time 777873916855 state 0x0 ref 1 00000100:00100000:2.0:1789760767.589753:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777874177271 time 777873916855 state 0x1 ref 1 00000100:00100000:2.0:1789760767.589791:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777874177271 time 777930826111 state 0x0 ref 1 00000100:00100000:2.0:1789760767.589806:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fadb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493798400:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760767.589874:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26054656, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xc807 00010000:00010000:2.0:1789760767.589887:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06419 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.589908:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:2.0:1789760767.589914:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.589922:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06419 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.589940:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06419 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760767.644064:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973680 is committed 00002000:00010000:2.0:1789760767.644207:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26054656 -> 26058752 00002000:00010000:2.0:1789760767.644213:0:13038:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760766 -> 1789760767 00002000:00010000:2.0:1789760767.644215:0:13038:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760766 -> 1789760767 00010000:00010000:2.0:1789760767.644229:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06419 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.644243:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06419 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.644252:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06419 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.644268:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760767.644273:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06419 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.644294:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06419 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.644315:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06419 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.644322:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06419 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.644455:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fadb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493798400:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54646us (54940us total) trans 4294973680 rc 0/0 00000100:00100000:2.0:1789760767.644466:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6391 00000100:00100000:2.0:1789760767.644478:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777874177271 time 777930826111 state 0x2 ref 0 00000100:00100000:0.0:1789760767.648259:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760767.648376:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493798528 00000100:00100000:1.0:1789760767.648386:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493798528 00000100:00100000:1.0:1789760767.648445:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777874177271 time 777930826111 state 0x0 ref 1 00000100:00100000:1.0:1789760767.648460:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777931086527 time 777930826111 state 0x1 ref 1 00000100:00100000:1.0:1789760767.648495:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777931086527 time 777989530447 state 0x0 ref 1 00000100:00100000:1.0:1789760767.648508:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493798528:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760767.648554:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26058752, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc807 00010000:00010000:1.0:1789760767.648564:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06420 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.648579:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760767.648584:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760767.648590:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06420 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.648604:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06420 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760767.717778:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973681 is committed 00002000:00010000:1.0:1789760767.717855:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26058752 -> 26062848 00010000:00010000:1.0:1789760767.717870:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06420 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.717883:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06420 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.717891:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06420 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.717898:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760767.717903:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06420 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.717912:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06420 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.717931:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06420 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.717939:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06420 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.718086:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493798528:12345-192.168.203.60@tcp:4:dd.0 Request processed in 69576us (69833us total) trans 4294973681 rc 0/0 00000100:00100000:1.0:1789760767.718096:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6392 00000100:00100000:1.0:1789760767.718106:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777931086527 time 777989530447 state 0x2 ref 0 00000100:00100000:0.0:1789760767.721655:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760767.721799:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493798656 00000100:00100000:3.0:1789760767.721811:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493798656 00000100:00100000:3.0:1789760767.721864:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777931086527 time 777989530447 state 0x0 ref 1 00000100:00100000:3.0:1789760767.721880:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777989790863 time 777989530447 state 0x1 ref 1 00000100:00100000:3.0:1789760767.721914:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777989790863 time 778062948905 state 0x0 ref 1 00000100:00100000:3.0:1789760767.721927:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493798656:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760767.722180:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26062848, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc807 00010000:00010000:3.0:1789760767.722190:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06427 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.722206:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=778s v=(0 0) 00010000:00010000:3.0:1789760767.722211:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.722217:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06427 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.722232:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06427 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.793114:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973682 is committed 00002000:00010000:0.0:1789760767.793483:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26062848 -> 26066944 00010000:00010000:0.0:1789760767.793539:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06427 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.793552:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06427 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.793562:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06427 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.793569:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760767.793574:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06427 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.793584:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06427 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.793603:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06427 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.793611:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06427 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.793749:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493798656:12345-192.168.203.60@tcp:4:dd.0 Request processed in 71820us (72100us total) trans 4294973682 rc 0/0 00000100:00100000:0.0:1789760767.793764:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6393 00000100:00100000:0.0:1789760767.793774:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777989790863 time 778062948905 state 0x2 ref 0 00000100:00100000:2.0:1789760767.805045:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760767.805209:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493798784 00000100:00100000:3.0:1789760767.805219:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493798784 00000100:00100000:3.0:1789760767.805277:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 777989790863 time 778062948905 state 0x0 ref 1 00000100:00100000:3.0:1789760767.805302:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778063209321 time 778062948905 state 0x1 ref 1 00000100:00100000:3.0:1789760767.805340:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778063209321 time 778146368572 state 0x0 ref 1 00000100:00100000:3.0:1789760767.805357:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493798784:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760767.805408:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26066944, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc807 00010000:00010000:3.0:1789760767.805417:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0642e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.805433:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=778s v=(0 0) 00010000:00010000:3.0:1789760767.805437:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.805442:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0642e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.805457:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0642e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760767.839618:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973683 is committed 00002000:00010000:0.0:1789760767.839725:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26066944 -> 26071040 00010000:00010000:0.0:1789760767.839743:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0642e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.839755:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0642e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.839763:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0642e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.839771:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760767.839776:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0642e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.839786:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0642e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.839807:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0642e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.841253:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e0642e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.845499:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493798784:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40129us (40449us total) trans 4294973683 rc 0/0 00000100:00100000:0.0:1789760767.845514:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6394 00000100:00100000:0.0:1789760767.845523:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778063209321 time 778146368572 state 0x2 ref 0 00000100:00100000:1.0:1789760767.849622:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760767.849711:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493798912 00000100:00100000:2.0:1789760767.849721:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493798912 00000100:00100000:2.0:1789760767.849772:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778063209321 time 778146368572 state 0x0 ref 1 00000100:00100000:2.0:1789760767.849788:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778146628988 time 778146368572 state 0x1 ref 1 00000100:00100000:2.0:1789760767.849827:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778146628988 time 778190862181 state 0x0 ref 1 00000100:00100000:2.0:1789760767.849840:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493798912:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760767.849886:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26071040, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc807 00010000:00010000:2.0:1789760767.849894:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06435 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.849909:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=778s v=(0 0) 00010000:00010000:2.0:1789760767.849914:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.849920:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06435 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.849934:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06435 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760767.889800:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26071040 -> 26075136 00010000:00010000:2.0:1789760767.889819:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06435 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.889837:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06435 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.889846:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06435 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.889853:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760767.889857:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06435 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.889866:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06435 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.889897:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06435 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.889905:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06435 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.890041:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493798912:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40197us (40423us total) trans 4294973684 rc 0/0 00000100:00100000:2.0:1789760767.890052:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6395 00000100:00100000:2.0:1789760767.890062:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778146628988 time 778190862181 state 0x2 ref 0 00000001:00080000:1.0:1789760767.890525:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973684 is committed 00000100:00100000:2.0:1789760767.898008:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760767.899552:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493799040 00000100:00100000:1.0:1789760767.899566:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493799040 00000100:00100000:1.0:1789760767.899629:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778146628988 time 778190862181 state 0x0 ref 1 00000100:00100000:1.0:1789760767.899646:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778191122597 time 778190862181 state 0x1 ref 1 00000100:00100000:1.0:1789760767.899684:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778191122597 time 778240719762 state 0x0 ref 1 00000100:00100000:1.0:1789760767.899698:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493799040:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760767.899747:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26075136, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc807 00010000:00010000:1.0:1789760767.899757:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0643c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.899771:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760767.899775:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760767.899781:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0643c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.899800:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0643c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760767.944872:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26075136 -> 26079232 00010000:00010000:2.0:1789760767.944893:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0643c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.944907:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0643c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.944916:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0643c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.944924:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760767.944929:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0643c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.944939:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0643c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.944979:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0643c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.944988:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0643c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.945172:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493799040:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45472us (47169us total) trans 4294973685 rc 0/0 00000100:00100000:2.0:1789760767.945184:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6396 00000100:00100000:2.0:1789760767.945195:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778191122597 time 778240719762 state 0x2 ref 0 00000001:00080000:3.0:1789760767.945197:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973685 is committed 00000100:00100000:2.0:1789760767.949494:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760767.949675:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493799168 00000100:00100000:0.0:1789760767.949684:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493799168 00000100:00100000:0.0:1789760767.949744:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778191122597 time 778240719762 state 0x0 ref 1 00000100:00100000:0.0:1789760767.949759:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778240980178 time 778240719762 state 0x1 ref 1 00000100:00100000:0.0:1789760767.949790:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778240980178 time 778290825610 state 0x0 ref 1 00000100:00100000:0.0:1789760767.949802:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493799168:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760767.949992:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26079232, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc807 00010000:00010000:0.0:1789760767.950002:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06443 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.950019:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=778s v=(0 0) 00010000:00010000:0.0:1789760767.950024:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.950030:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06443 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.950044:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06443 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.993115:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973686 is committed 00002000:00010000:0.0:1789760767.993214:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26079232 -> 26083328 00010000:00010000:0.0:1789760767.993231:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06443 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.993244:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06443 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.993254:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06443 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.993262:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760767.993267:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06443 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.993277:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06443 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.993305:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06443 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.993314:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06443 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.993720:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493799168:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43911us (44228us total) trans 4294973686 rc 0/0 00000100:00100000:0.0:1789760767.993735:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6397 00000100:00100000:0.0:1789760767.993744:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778240980178 time 778290825610 state 0x2 ref 0 00000100:00100000:1.0:1789760767.997752:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760767.997861:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493799296 00000100:00100000:0.0:1789760767.997870:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493799296 00000100:00100000:0.0:1789760767.997922:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778240980178 time 778290825610 state 0x0 ref 1 00000100:00100000:0.0:1789760767.997937:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778291086026 time 778290825610 state 0x1 ref 1 00000100:00100000:0.0:1789760767.998027:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778291086026 time 778339061608 state 0x0 ref 1 00000100:00100000:0.0:1789760767.998039:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493799296:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760767.998090:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26083328, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc807 00010000:00010000:0.0:1789760767.998100:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0644a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.998115:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=778s v=(0 0) 00010000:00010000:0.0:1789760767.998120:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.998127:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0644a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.998142:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0644a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760768.027441:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973687 is committed 00002000:00010000:0.0:1789760768.027557:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26083328 -> 26087424 00010000:00010000:0.0:1789760768.027576:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0644a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.027589:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0644a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.027598:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0644a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.027605:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760768.027610:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0644a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.027622:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0644a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.027641:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0644a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.027649:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0644a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.027747:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493799296:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29705us (30000us total) trans 4294973687 rc 0/0 00000100:00100000:0.0:1789760768.027758:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6398 00000100:00100000:0.0:1789760768.027767:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778291086026 time 778339061608 state 0x2 ref 0 00000100:00100000:1.0:1789760768.031125:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760768.031265:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493799424 00000100:00100000:2.0:1789760768.031275:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493799424 00000100:00100000:2.0:1789760768.031339:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778291086026 time 778339061608 state 0x0 ref 1 00000100:00100000:2.0:1789760768.031353:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778339322024 time 778339061608 state 0x1 ref 1 00000100:00100000:0.0:1789760768.031784:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778339322024 time 778372816222 state 0x0 ref 1 00000100:00100000:0.0:1789760768.031802:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493799424:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760768.031852:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26087424, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc807 00010000:00010000:0.0:1789760768.031860:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06451 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.031875:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=778s v=(0 0) 00010000:00010000:0.0:1789760768.031880:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.031886:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06451 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.031900:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06451 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760768.064169:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973688 is committed 00002000:00010000:0.0:1789760768.064473:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26087424 -> 26091520 00010000:00010000:0.0:1789760768.064488:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06451 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.064503:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06451 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.064512:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06451 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.064519:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760768.064524:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06451 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.064533:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06451 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.064551:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06451 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.064559:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06451 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.064685:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493799424:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32882us (33563us total) trans 4294973688 rc 0/0 00000100:00100000:0.0:1789760768.064695:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6399 00000100:00100000:0.0:1789760768.064704:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778339322024 time 778372816222 state 0x2 ref 0 00000100:00100000:0.0:1789760768.070936:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760768.071232:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493799552 00000100:00100000:2.0:1789760768.071242:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493799552 00000100:00100000:3.0:1789760768.071414:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778339322024 time 778372816222 state 0x0 ref 1 00000100:00100000:3.0:1789760768.071429:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778373076638 time 778372816222 state 0x1 ref 1 00000100:00100000:3.0:1789760768.071456:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778373076638 time 778412492511 state 0x0 ref 1 00000100:00100000:3.0:1789760768.071468:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493799552:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760768.071515:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26091520, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc807 00010000:00010000:3.0:1789760768.071525:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06458 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.071540:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=778s v=(0 0) 00010000:00010000:3.0:1789760768.071545:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.071551:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06458 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.071564:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06458 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760768.102896:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760768.103153:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493799808 00000100:00100000:2.0:1789760768.103163:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493799808 00000100:00100000:2.0:1789760768.103234:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 507 00000100:00100000:2.0:1789760768.103245:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2370:x1876699493799808:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:2.0:1789760768.103327:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2370:x1876699493799808:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 84us (441us total) trans 0 rc 0/0 00000100:00080000:1.0:1789760768.103722:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760768.103870:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 507 00002000:00010000:2.0:1789760768.105934:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26091520 -> 26095616 00010000:00010000:2.0:1789760768.105955:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06458 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.106000:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06458 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.106010:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06458 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.106023:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760768.106028:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06458 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.106038:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06458 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.106059:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06458 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.106067:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06458 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.106161:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493799552:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34689us (35227us total) trans 4294973689 rc 0/0 00000100:00100000:2.0:1789760768.106171:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6400 00000100:00100000:2.0:1789760768.106181:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778373076638 time 778412492511 state 0x2 ref 0 00000001:00080000:3.0:1789760768.107433:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973689 is committed 00000100:00100000:1.0:1789760768.107646:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760768.107687:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760768.107803:0:10519:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493799936 00000100:00100000:2.0:1789760768.107812:0:10519:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493799936 00000100:00100000:2.0:1789760768.107865:0:10519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 508 00000100:00100000:2.0:1789760768.107874:0:10519:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48de680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2369:x1876699493799936:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:2.0:1789760768.107932:0:10519:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48de680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2369:x1876699493799936:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 60us (245us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760768.107940:0:10519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 508 00000100:00100000:3.0:1789760768.108546:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493799680 00000100:00100000:3.0:1789760768.108553:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493799680 00000100:00100000:3.0:1789760768.108599:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 409 00000100:00100000:3.0:1789760768.108608:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d4b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2369:x1876699493799680:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:3.0:1789760768.108644:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d4b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2369:x1876699493799680:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 37us (1002us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760768.108650:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 409 00000100:00080000:0.0:1789760768.109092:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760768.114887:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760768.114990:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493800064 00000100:00100000:2.0:1789760768.115001:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493800064 00000100:00100000:2.0:1789760768.115056:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778373076638 time 778412492511 state 0x0 ref 1 00000100:00100000:2.0:1789760768.115071:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778412752927 time 778412492511 state 0x1 ref 1 00000100:00100000:2.0:1789760768.115101:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778412752927 time 778456136635 state 0x0 ref 1 00000100:00100000:2.0:1789760768.115114:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493800064:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760768.115168:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26095616, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc807 00010000:00010000:2.0:1789760768.115176:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0645f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.115191:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=778s v=(0 0) 00010000:00010000:2.0:1789760768.115195:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.115202:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0645f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.115215:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0645f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760768.158147:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973690 is committed 00002000:00010000:0.0:1789760768.158249:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26095616 -> 26099712 00010000:00010000:0.0:1789760768.158263:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0645f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.158276:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0645f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.158296:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0645f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.158304:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760768.158309:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0645f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.158318:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0645f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.158349:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0645f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.158356:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0645f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.158450:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493800064:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43333us (43569us total) trans 4294973690 rc 0/0 00000100:00100000:0.0:1789760768.158461:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6401 00000100:00100000:0.0:1789760768.158470:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778412752927 time 778456136635 state 0x2 ref 0 00000100:00100000:3.0:1789760768.164609:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760768.166426:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493800192 00000100:00100000:0.0:1789760768.166439:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493800192 00000100:00100000:0.0:1789760768.166502:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778412752927 time 778456136635 state 0x0 ref 1 00000100:00100000:0.0:1789760768.166519:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778456397051 time 778456136635 state 0x1 ref 1 00000100:00100000:0.0:1789760768.166555:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778456397051 time 778507590359 state 0x0 ref 1 00000100:00100000:0.0:1789760768.166572:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493800192:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760768.166621:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26099712, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc807 00010000:00010000:0.0:1789760768.166630:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06466 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.166648:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=778s v=(0 0) 00010000:00010000:0.0:1789760768.166652:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.166658:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06466 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.166672:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06466 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760768.197343:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973691 is committed 00002000:00010000:2.0:1789760768.197529:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26099712 -> 26103808 00010000:00010000:2.0:1789760768.197546:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06466 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.197560:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06466 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.197575:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06466 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.197582:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760768.197587:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06466 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.197597:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06466 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.197616:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06466 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.197624:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06466 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.197717:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493800192:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31142us (33112us total) trans 4294973691 rc 0/0 00000100:00100000:2.0:1789760768.197727:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6402 00000100:00100000:2.0:1789760768.197736:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778456397051 time 778507590359 state 0x2 ref 0 00000100:00100000:1.0:1789760768.203318:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760768.203417:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493800320 00000100:00100000:0.0:1789760768.203427:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493800320 00000100:00100000:0.0:1789760768.203481:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778456397051 time 778507590359 state 0x0 ref 1 00000100:00100000:0.0:1789760768.203497:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778507850775 time 778507590359 state 0x1 ref 1 00000100:00100000:0.0:1789760768.203530:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778507850775 time 778544565064 state 0x0 ref 1 00000100:00100000:0.0:1789760768.203543:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493800320:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760768.203592:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26103808, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc807 00010000:00010000:0.0:1789760768.203601:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0646d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.203622:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=778s v=(0 0) 00010000:00010000:0.0:1789760768.203626:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.203632:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0646d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.203646:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0646d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760768.235573:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973692 is committed 00002000:00010000:0.0:1789760768.235676:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26103808 -> 26107904 00010000:00010000:0.0:1789760768.235692:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0646d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.235705:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0646d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.235714:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0646d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.235722:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760768.235726:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0646d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.235736:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0646d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.235756:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0646d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.235776:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0646d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.236109:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493800320:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32563us (32796us total) trans 4294973692 rc 0/0 00000100:00100000:0.0:1789760768.236121:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6403 00000100:00100000:0.0:1789760768.236131:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778507850775 time 778544565064 state 0x2 ref 0 00000100:00100000:0.0:1789760768.241194:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760768.241294:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493800448 00000100:00100000:2.0:1789760768.241305:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493800448 00000100:00100000:2.0:1789760768.241373:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778507850775 time 778544565064 state 0x0 ref 1 00000100:00100000:2.0:1789760768.241393:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778544825480 time 778544565064 state 0x1 ref 1 00000100:00100000:2.0:1789760768.241425:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778544825480 time 778582460094 state 0x0 ref 1 00000100:00100000:2.0:1789760768.241437:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493800448:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760768.241490:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26107904, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc807 00010000:00010000:2.0:1789760768.241499:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06474 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.241515:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=778s v=(0 0) 00010000:00010000:2.0:1789760768.241520:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.241526:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06474 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.241540:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06474 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760768.276775:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973693 is committed 00002000:00010000:2.0:1789760768.277019:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26107904 -> 26112000 00010000:00010000:2.0:1789760768.277034:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06474 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.277048:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06474 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.277057:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06474 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.277069:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760768.277075:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06474 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.277084:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06474 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.277104:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06474 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.277112:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06474 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.277202:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493800448:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35762us (36014us total) trans 4294973693 rc 0/0 00000100:00100000:2.0:1789760768.277213:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6404 00000100:00100000:2.0:1789760768.277223:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778544825480 time 778582460094 state 0x2 ref 0 00000100:00100000:2.0:1789760768.286903:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760768.287008:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493800576 00000100:00100000:0.0:1789760768.287018:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493800576 00000100:00100000:0.0:1789760768.287076:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778544825480 time 778582460094 state 0x0 ref 1 00000100:00100000:0.0:1789760768.287092:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778582720510 time 778582460094 state 0x1 ref 1 00000100:00100000:1.0:1789760768.287206:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778582720510 time 778628237920 state 0x0 ref 1 00000100:00100000:1.0:1789760768.287223:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493800576:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760768.287682:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26112000, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc807 00010000:00010000:1.0:1789760768.287695:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0647b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.287712:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760768.287716:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760768.287723:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0647b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.287743:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0647b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.352038:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973694 is committed 00002000:00010000:2.0:1789760768.352168:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26112000 -> 26116096 00010000:00010000:2.0:1789760768.352183:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0647b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.352196:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0647b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.352210:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0647b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.352217:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760768.352222:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0647b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.352231:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0647b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.352250:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0647b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.352258:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0647b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.352464:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493800576:12345-192.168.203.60@tcp:4:dd.0 Request processed in 65240us (65566us total) trans 4294973694 rc 0/0 00000100:00100000:2.0:1789760768.352476:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6405 00000100:00100000:2.0:1789760768.352487:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778582720510 time 778628237920 state 0x2 ref 0 00000100:00100000:1.0:1789760768.361134:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760768.361238:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493800704 00000100:00100000:0.0:1789760768.361248:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493800704 00000100:00100000:0.0:1789760768.361310:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778582720510 time 778628237920 state 0x0 ref 1 00000100:00100000:0.0:1789760768.361326:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778628498336 time 778628237920 state 0x1 ref 1 00000100:00100000:0.0:1789760768.361357:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778628498336 time 778702392876 state 0x0 ref 1 00000100:00100000:0.0:1789760768.361371:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493800704:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760768.361421:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26116096, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc807 00010000:00010000:0.0:1789760768.361432:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06482 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.361447:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=778s v=(0 0) 00010000:00010000:0.0:1789760768.361452:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.361458:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06482 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.361471:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06482 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760768.389798:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26116096 -> 26120192 00010000:00010000:1.0:1789760768.389822:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06482 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.389835:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06482 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.389845:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06482 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.389853:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760768.389857:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06482 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.389866:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06482 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.389889:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06482 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.389898:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06482 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760768.390028:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493800704:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28655us (28899us total) trans 4294973695 rc 0/0 00000100:00100000:1.0:1789760768.390039:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6406 00000100:00100000:1.0:1789760768.390049:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778628498336 time 778702392876 state 0x2 ref 0 00000001:00080000:3.0:1789760768.390231:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973695 is committed 00000100:00100000:0.0:1789760768.396455:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760768.396696:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493800832 00000100:00100000:3.0:1789760768.396705:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493800832 00000100:00100000:3.0:1789760768.396836:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778628498336 time 778702392876 state 0x0 ref 1 00000100:00100000:3.0:1789760768.396853:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778702653292 time 778702392876 state 0x1 ref 1 00000100:00100000:3.0:1789760768.396884:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778702653292 time 778737919554 state 0x0 ref 1 00000100:00100000:3.0:1789760768.396899:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493800832:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760768.396946:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26120192, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc807 00010000:00010000:3.0:1789760768.396954:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06489 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.397126:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=778s v=(0 0) 00010000:00010000:3.0:1789760768.397130:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.397136:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06489 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.397150:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06489 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760768.435896:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26120192 -> 26124288 00010000:00010000:1.0:1789760768.435916:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06489 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.435931:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06489 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.435939:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06489 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.435945:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760768.435950:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06489 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.435989:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06489 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.436011:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06489 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.436023:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06489 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760768.436114:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493800832:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39211us (39663us total) trans 4294973696 rc 0/0 00000100:00100000:1.0:1789760768.436124:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6407 00000100:00100000:1.0:1789760768.436134:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778702653292 time 778737919554 state 0x2 ref 0 00000001:00080000:3.0:1789760768.436354:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973696 is committed 00000100:00100000:2.0:1789760768.443879:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760768.444802:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493800960 00000100:00100000:3.0:1789760768.444814:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493800960 00000100:00100000:3.0:1789760768.444875:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778702653292 time 778737919554 state 0x0 ref 1 00000100:00100000:3.0:1789760768.444892:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778738179970 time 778737919554 state 0x1 ref 1 00000100:00100000:3.0:1789760768.444990:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778738179970 time 778786025134 state 0x0 ref 1 00000100:00100000:3.0:1789760768.445013:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493800960:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760768.445073:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26124288, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc807 00010000:00010000:3.0:1789760768.445082:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06490 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.445099:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=778s v=(0 0) 00010000:00010000:3.0:1789760768.445104:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.445110:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06490 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.445125:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06490 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760768.481807:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26124288 -> 26128384 00010000:00010000:1.0:1789760768.481825:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06490 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.481839:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06490 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.481848:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06490 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.481854:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760768.481859:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06490 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.481868:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06490 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.481892:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06490 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.481900:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06490 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760768.481957:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973697 is committed 00000100:00100000:1.0:1789760768.482027:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493800960:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37011us (38153us total) trans 4294973697 rc 0/0 00000100:00100000:1.0:1789760768.482036:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6408 00000100:00100000:1.0:1789760768.482046:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778738179970 time 778786025134 state 0x2 ref 0 00000100:00100000:0.0:1789760768.487448:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760768.487606:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493801088 00000100:00100000:2.0:1789760768.487616:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493801088 00000100:00100000:2.0:1789760768.487669:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778738179970 time 778786025134 state 0x0 ref 1 00000100:00100000:2.0:1789760768.487687:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778786285550 time 778786025134 state 0x1 ref 1 00000100:00100000:2.0:1789760768.487714:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778786285550 time 778828750642 state 0x0 ref 1 00000100:00100000:2.0:1789760768.487727:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493801088:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760768.487786:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26128384, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc807 00010000:00010000:2.0:1789760768.487796:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06497 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.487812:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=778s v=(0 0) 00010000:00010000:2.0:1789760768.487816:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.487822:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06497 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.487836:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06497 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760768.515467:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26128384 -> 26132480 00010000:00010000:3.0:1789760768.515490:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06497 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.515504:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06497 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.515512:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06497 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.515520:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760768.515524:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06497 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.515534:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06497 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.515560:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06497 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.515567:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06497 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.515659:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493801088:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27929us (28214us total) trans 4294973698 rc 0/0 00000100:00100000:3.0:1789760768.515670:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6409 00000100:00100000:3.0:1789760768.515679:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778786285550 time 778828750642 state 0x2 ref 0 00000001:00080000:3.0:1789760768.515723:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973698 is committed 00000100:00100000:3.0:1789760768.519311:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760768.519397:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493801216 00000100:00100000:1.0:1789760768.519408:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493801216 00000100:00100000:1.0:1789760768.519464:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778786285550 time 778828750642 state 0x0 ref 1 00000100:00100000:1.0:1789760768.519478:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778829011058 time 778828750642 state 0x1 ref 1 00000100:00100000:1.0:1789760768.519512:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778829011058 time 778860547526 state 0x0 ref 1 00000100:00100000:1.0:1789760768.519524:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb15450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493801216:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760768.519571:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26132480, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc807 00010000:00010000:1.0:1789760768.519580:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0649e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.519594:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760768.519598:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760768.519605:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0649e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.519617:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0649e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760768.546291:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26132480 -> 26136576 00010000:00010000:1.0:1789760768.546311:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0649e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.546322:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0649e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.546340:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0649e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.546347:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760768.546351:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0649e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.546360:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0649e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.546380:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0649e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.546389:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e0649e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760768.546476:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb15450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493801216:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26949us (27170us total) trans 4294973699 rc 0/0 00000100:00100000:1.0:1789760768.546486:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6410 00000100:00100000:1.0:1789760768.546495:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778829011058 time 778860547526 state 0x2 ref 0 00000001:00080000:1.0:1789760768.546654:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973699 is committed 00000100:00100000:2.0:1789760768.550167:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760768.550253:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493801344 00000100:00100000:1.0:1789760768.550263:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493801344 00000100:00100000:1.0:1789760768.550325:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778829011058 time 778860547526 state 0x0 ref 1 00000100:00100000:1.0:1789760768.550348:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778860807942 time 778860547526 state 0x1 ref 1 00000100:00100000:1.0:1789760768.550376:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778860807942 time 778891411563 state 0x0 ref 1 00000100:00100000:1.0:1789760768.550388:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493801344:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760768.550434:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26136576, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc807 00010000:00010000:1.0:1789760768.550441:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e064a5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.550456:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760768.550460:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760768.550466:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e064a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.550479:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e064a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.572584:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760768.576829:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973700 is committed 00002000:00010000:0.0:1789760768.577012:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26136576 -> 26140672 00010000:00010000:0.0:1789760768.577026:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e064a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.577037:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e064a5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.577046:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e064a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.577053:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760768.577057:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e064a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.577071:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e064a5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.577090:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e064a5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.577098:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e064a5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.577176:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493801344:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26785us (27014us total) trans 4294973700 rc 0/0 00000100:00100000:0.0:1789760768.577186:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6411 00000100:00100000:0.0:1789760768.577195:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778860807942 time 778891411563 state 0x2 ref 0 00000100:00100000:0.0:1789760768.581128:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760768.581285:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493801472 00000100:00100000:1.0:1789760768.581295:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493801472 00000100:00100000:1.0:1789760768.581409:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778860807942 time 778891411563 state 0x0 ref 1 00000100:00100000:1.0:1789760768.581423:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778891671979 time 778891411563 state 0x1 ref 1 00000100:00100000:1.0:1789760768.581488:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778891671979 time 778922522519 state 0x0 ref 1 00000100:00100000:1.0:1789760768.581501:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493801472:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760768.581547:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26140672, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xc807 00010000:00010000:1.0:1789760768.581555:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e064ac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.581569:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760768.581573:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760768.581580:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e064ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.581593:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e064ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.610359:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973701 is committed 00002000:00010000:1.0:1789760768.610518:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26140672 -> 26144768 00002000:00010000:1.0:1789760768.610523:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760767 -> 1789760768 00002000:00010000:1.0:1789760768.610526:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760767 -> 1789760768 00010000:00010000:1.0:1789760768.610542:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e064ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.610555:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e064ac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.610564:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e064ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.610571:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760768.610576:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e064ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.610586:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e064ac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.610606:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e064ac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.610614:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e064ac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760768.610714:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493801472:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29210us (29589us total) trans 4294973701 rc 0/0 00000100:00100000:1.0:1789760768.610724:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6412 00000100:00100000:1.0:1789760768.610733:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778891671979 time 778922522519 state 0x2 ref 0 00000100:00100000:2.0:1789760768.614191:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760768.614288:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493801600 00000100:00100000:3.0:1789760768.614296:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493801600 00000100:00100000:3.0:1789760768.614408:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778891671979 time 778922522519 state 0x0 ref 1 00000100:00100000:3.0:1789760768.614423:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778922782935 time 778922522519 state 0x1 ref 1 00000100:00100000:3.0:1789760768.614457:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778922782935 time 778955492519 state 0x0 ref 1 00000100:00100000:3.0:1789760768.614469:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493801600:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760768.614515:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26144768, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc807 00010000:00010000:3.0:1789760768.614525:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064b3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.614539:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=778s v=(0 0) 00010000:00010000:3.0:1789760768.614543:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.614549:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.614561:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760768.644610:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973702 is committed 00002000:00010000:3.0:1789760768.644716:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26144768 -> 26148864 00010000:00010000:3.0:1789760768.644731:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.644744:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064b3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.644752:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.644834:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760768.644840:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.644848:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064b3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.644867:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064b3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.644876:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064b3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.645009:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493801600:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30536us (30823us total) trans 4294973702 rc 0/0 00000100:00100000:3.0:1789760768.645019:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6413 00000100:00100000:3.0:1789760768.645029:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778922782935 time 778955492519 state 0x2 ref 0 00000100:00100000:3.0:1789760768.648627:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760768.648741:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493801728 00000100:00100000:2.0:1789760768.648752:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493801728 00000100:00100000:2.0:1789760768.648866:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778922782935 time 778955492519 state 0x0 ref 1 00000100:00100000:2.0:1789760768.648881:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778955752935 time 778955492519 state 0x1 ref 1 00000100:00100000:2.0:1789760768.649007:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778955752935 time 778990041681 state 0x0 ref 1 00000100:00100000:2.0:1789760768.649021:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493801728:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760768.649072:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26148864, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc807 00010000:00010000:2.0:1789760768.649081:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e064ba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.649097:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=778s v=(0 0) 00010000:00010000:2.0:1789760768.649101:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.649108:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e064ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.649121:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e064ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760768.678365:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26148864 -> 26152960 00010000:00010000:2.0:1789760768.678385:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e064ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.678398:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e064ba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.678407:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e064ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.678415:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760768.678421:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e064ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.678430:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e064ba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.678451:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e064ba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.678458:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e064ba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.678547:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493801728:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29524us (29924us total) trans 4294973703 rc 0/0 00000100:00100000:2.0:1789760768.678557:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6414 00000100:00100000:2.0:1789760768.678567:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778955752935 time 778990041681 state 0x2 ref 0 00000001:00080000:1.0:1789760768.681018:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973703 is committed 00000100:00100000:1.0:1789760768.681900:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760768.682006:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493801856 00000100:00100000:2.0:1789760768.682016:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493801856 00000100:00100000:2.0:1789760768.682069:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778955752935 time 778990041681 state 0x0 ref 1 00000100:00100000:2.0:1789760768.682083:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778990302097 time 778990041681 state 0x1 ref 1 00000100:00100000:2.0:1789760768.682113:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778990302097 time 779023149277 state 0x0 ref 1 00000100:00100000:2.0:1789760768.682127:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493801856:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760768.682174:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26152960, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc807 00010000:00010000:2.0:1789760768.682183:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e064c1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.682198:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760768.682203:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.682209:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e064c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.682224:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e064c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760768.739568:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26152960 -> 26157056 00010000:00010000:3.0:1789760768.739588:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e064c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.739601:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e064c1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.739610:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e064c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.739617:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760768.739622:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e064c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.739631:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e064c1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.739656:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e064c1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.739665:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e064c1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.739766:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493801856:12345-192.168.203.60@tcp:4:dd.0 Request processed in 57629us (57858us total) trans 4294973704 rc 0/0 00000100:00100000:3.0:1789760768.739777:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6415 00000100:00100000:3.0:1789760768.739787:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778990302097 time 779023149277 state 0x2 ref 0 00000001:00080000:1.0:1789760768.739921:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973704 is committed 00000100:00100000:2.0:1789760768.743488:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760768.743588:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493801984 00000100:00100000:1.0:1789760768.743598:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493801984 00000100:00100000:1.0:1789760768.743653:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 778990302097 time 779023149277 state 0x0 ref 1 00000100:00100000:1.0:1789760768.743667:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779023409693 time 779023149277 state 0x1 ref 1 00000100:00100000:1.0:1789760768.743696:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779023409693 time 779084731829 state 0x0 ref 1 00000100:00100000:1.0:1789760768.743710:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493801984:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760768.743756:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26157056, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc807 00010000:00010000:1.0:1789760768.743777:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064c8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.743791:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760768.743796:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760768.743802:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.743816:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760768.782739:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26157056 -> 26161152 00010000:00010000:1.0:1789760768.782767:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.782785:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.782794:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.782802:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760768.782808:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.782817:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.782836:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064c8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.782843:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064c8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760768.782940:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493801984:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39228us (39457us total) trans 4294973705 rc 0/0 00000100:00100000:1.0:1789760768.782951:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6416 00000100:00100000:1.0:1789760768.782990:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779023409693 time 779084731829 state 0x2 ref 0 00000001:00080000:3.0:1789760768.787561:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973705 is committed 00000100:00100000:2.0:1789760768.787680:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760768.787954:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493802112 00000100:00100000:3.0:1789760768.788135:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493802112 00000100:00100000:3.0:1789760768.788191:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779023409693 time 779084731829 state 0x0 ref 1 00000100:00100000:3.0:1789760768.788207:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779084992245 time 779084731829 state 0x1 ref 1 00000100:00100000:3.0:1789760768.788282:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779084992245 time 779129308895 state 0x0 ref 1 00000100:00100000:3.0:1789760768.788296:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493802112:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760768.788473:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26161152, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc807 00010000:00010000:3.0:1789760768.788482:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064cf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.788504:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760768.788509:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.788515:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.788529:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760768.820667:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973706 is committed 00002000:00010000:2.0:1789760768.821027:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26161152 -> 26165248 00010000:00010000:2.0:1789760768.821042:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.821055:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.821063:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.821070:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760768.821075:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.821090:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064cf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.821109:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064cf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.821116:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064cf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.821311:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493802112:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33012us (33633us total) trans 4294973706 rc 0/0 00000100:00100000:2.0:1789760768.821322:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6417 00000100:00100000:2.0:1789760768.821340:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779084992245 time 779129308895 state 0x2 ref 0 00000100:00100000:2.0:1789760768.825513:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760768.828247:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493802240 00000100:00100000:1.0:1789760768.828260:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493802240 00000100:00100000:1.0:1789760768.828486:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779084992245 time 779129308895 state 0x0 ref 1 00000100:00100000:1.0:1789760768.828502:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779129569311 time 779129308895 state 0x1 ref 1 00000100:00100000:1.0:1789760768.828532:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779129569311 time 779169567497 state 0x0 ref 1 00000100:00100000:1.0:1789760768.828545:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493802240:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760768.828595:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26165248, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc807 00010000:00010000:1.0:1789760768.828603:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064d6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.828617:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760768.828620:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760768.828626:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.828639:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.890236:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc33ece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519257856:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760768.890347:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760768.890424:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519257856 00000100:00100000:2.0:1789760768.890433:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519257856 00000100:00100000:2.0:1789760768.890497:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 210 00000100:00100000:2.0:1789760768.890509:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519257856:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760768.890535:0:6690:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527630336 pending 1703936 free 3832545280 avail 3827302400 00000100:00100000:2.0:1789760768.890575:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519257856:12345-0@lo:13:osp-pre-0-0.0 Request processed in 70us (229us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760768.890583:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 210 00000100:00080000:1.0:1789760768.890640:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760768.890715:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc33ece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519257856:0@lo:13:osp-pre-0-0.0 00000001:00080000:3.0:1789760768.891379:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973707 is committed 00002000:00010000:2.0:1789760768.891510:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26165248 -> 26169344 00010000:00010000:2.0:1789760768.891522:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.891536:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064d6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.891545:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.891552:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760768.891556:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.891565:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064d6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.891583:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064d6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.891590:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064d6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.891657:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493802240:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63113us (66152us total) trans 4294973707 rc 0/0 00000100:00100000:2.0:1789760768.891666:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6418 00000100:00100000:2.0:1789760768.891675:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779129569311 time 779169567497 state 0x2 ref 0 00000100:00100000:0.0:1789760768.894372:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760768.894680:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493802368 00000100:00100000:2.0:1789760768.894690:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493802368 00000100:00100000:2.0:1789760768.894746:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779129569311 time 779169567497 state 0x0 ref 1 00000100:00100000:2.0:1789760768.894761:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779169827913 time 779169567497 state 0x1 ref 1 00000100:00100000:2.0:1789760768.896020:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779169827913 time 779237049525 state 0x0 ref 1 00000100:00100000:2.0:1789760768.896045:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493802368:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760768.896098:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26169344, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc807 00010000:00010000:2.0:1789760768.896107:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e064dd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.896122:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760768.896126:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.896132:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e064dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.896146:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e064dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760768.947139:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26169344 -> 26173440 00010000:00010000:3.0:1789760768.947167:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e064dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.947180:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e064dd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.947189:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e064dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.947197:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760768.947201:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e064dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.947214:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e064dd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.947234:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e064dd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.947242:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e064dd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.947346:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493802368:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51307us (52978us total) trans 4294973708 rc 0/0 00000100:00100000:3.0:1789760768.947356:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6419 00000100:00100000:3.0:1789760768.947365:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779169827913 time 779237049525 state 0x2 ref 0 00000001:00080000:3.0:1789760768.947404:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973708 is committed 00000100:00100000:3.0:1789760768.950219:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760768.950296:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493802496 00000100:00100000:2.0:1789760768.950307:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493802496 00000100:00100000:2.0:1789760768.950411:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779169827913 time 779237049525 state 0x0 ref 1 00000100:00100000:2.0:1789760768.950425:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779237309941 time 779237049525 state 0x1 ref 1 00000100:00100000:2.0:1789760768.950461:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779237309941 time 779291496699 state 0x0 ref 1 00000100:00100000:2.0:1789760768.950479:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4529c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493802496:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760768.950525:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26173440, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc807 00010000:00010000:2.0:1789760768.950534:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e064e4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.950549:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760768.950554:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.950560:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e064e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.950571:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e064e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.985625:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973709 is committed 00002000:00010000:3.0:1789760768.985766:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26173440 -> 26177536 00010000:00010000:3.0:1789760768.985788:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e064e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.985801:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e064e4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.985809:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e064e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.985822:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760768.985827:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e064e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.985836:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e064e4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.985854:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e064e4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.985861:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e064e4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.986038:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4529c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493802496:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35558us (35824us total) trans 4294973709 rc 0/0 00000100:00100000:3.0:1789760768.986049:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6420 00000100:00100000:3.0:1789760768.986059:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779237309941 time 779291496699 state 0x2 ref 0 00000100:00100000:2.0:1789760768.992146:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760768.992245:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493802624 00000100:00100000:3.0:1789760768.992257:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493802624 00000100:00100000:3.0:1789760768.992463:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779237309941 time 779291496699 state 0x0 ref 1 00000100:00100000:3.0:1789760768.992478:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779291757115 time 779291496699 state 0x1 ref 1 00000100:00100000:3.0:1789760768.992541:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779291757115 time 779333576467 state 0x0 ref 1 00000100:00100000:3.0:1789760768.992558:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493802624:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760768.992607:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26177536, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc807 00010000:00010000:3.0:1789760768.992615:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e064eb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.992631:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760768.992635:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.992641:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e064eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.992654:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e064eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.024410:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973710 is committed 00002000:00010000:2.0:1789760769.024615:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26177536 -> 26181632 00010000:00010000:2.0:1789760769.024630:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e064eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.024645:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e064eb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.024653:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e064eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.024659:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760769.024664:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e064eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.024673:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e064eb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.024692:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e064eb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.024699:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e064eb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.024864:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493802624:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32302us (32721us total) trans 4294973710 rc 0/0 00000100:00100000:2.0:1789760769.024874:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6421 00000100:00100000:2.0:1789760769.024884:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779291757115 time 779333576467 state 0x2 ref 0 00000100:00100000:3.0:1789760769.030235:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760769.031240:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493802752 00000100:00100000:1.0:1789760769.031266:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493802752 00000100:00100000:1.0:1789760769.031354:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779291757115 time 779333576467 state 0x0 ref 1 00000100:00100000:1.0:1789760769.031370:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779333836883 time 779333576467 state 0x1 ref 1 00000100:00100000:1.0:1789760769.031403:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779333836883 time 779372438293 state 0x0 ref 1 00000100:00100000:1.0:1789760769.031418:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493802752:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760769.031466:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26181632, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc807 00010000:00010000:1.0:1789760769.031475:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064f2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.031492:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760769.031497:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.031504:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.031518:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760769.032739:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760769.097512:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973711 is committed 00002000:00010000:0.0:1789760769.097611:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26181632 -> 26185728 00010000:00010000:0.0:1789760769.097629:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.097642:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064f2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.097671:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.097679:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760769.097684:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.097694:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064f2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.097714:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064f2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.097722:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e064f2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.098455:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493802752:12345-192.168.203.60@tcp:4:dd.0 Request processed in 67033us (68222us total) trans 4294973711 rc 0/0 00000100:00100000:0.0:1789760769.098466:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6422 00000100:00100000:0.0:1789760769.098475:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779333836883 time 779372438293 state 0x2 ref 0 00000100:00100000:2.0:1789760769.102536:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760769.102630:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493802880 00000100:00100000:3.0:1789760769.102640:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493802880 00000100:00100000:3.0:1789760769.102696:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779333836883 time 779372438293 state 0x0 ref 1 00000100:00100000:3.0:1789760769.102711:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779372698709 time 779372438293 state 0x1 ref 1 00000100:00100000:3.0:1789760769.102832:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779372698709 time 779443866401 state 0x0 ref 1 00000100:00100000:3.0:1789760769.102845:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493802880:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760769.102889:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26185728, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc807 00010000:00010000:3.0:1789760769.102897:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064f9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.103042:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760769.103047:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.103054:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.103067:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.132319:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973712 is committed 00002000:00010000:2.0:1789760769.132474:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26185728 -> 26189824 00010000:00010000:2.0:1789760769.132490:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.132504:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064f9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.132512:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.132525:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760769.132529:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.132538:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064f9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.132557:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064f9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.132565:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e064f9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.132660:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493802880:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29812us (30128us total) trans 4294973712 rc 0/0 00000100:00100000:2.0:1789760769.132672:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6423 00000100:00100000:2.0:1789760769.132681:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779372698709 time 779443866401 state 0x2 ref 0 00000100:00100000:1.0:1789760769.137229:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760769.137312:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493803008 00000100:00100000:3.0:1789760769.137322:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493803008 00000100:00100000:3.0:1789760769.137391:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779372698709 time 779443866401 state 0x0 ref 1 00000100:00100000:3.0:1789760769.137407:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779444126817 time 779443866401 state 0x1 ref 1 00000100:00100000:3.0:1789760769.137442:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779444126817 time 779478476497 state 0x0 ref 1 00000100:00100000:3.0:1789760769.137455:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493803008:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760769.137510:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26189824, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc807 00010000:00010000:3.0:1789760769.137521:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06500 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.137537:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760769.137541:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.137548:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06500 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.137562:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06500 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760769.178458:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26189824 -> 26193920 00010000:00010000:2.0:1789760769.178478:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06500 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.178490:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06500 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.178505:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06500 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.178512:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760769.178517:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06500 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.178525:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06500 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.178543:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06500 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.178549:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06500 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.178640:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493803008:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41182us (41415us total) trans 4294973713 rc 0/0 00000100:00100000:2.0:1789760769.178650:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6424 00000100:00100000:2.0:1789760769.178658:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779444126817 time 779478476497 state 0x2 ref 0 00000001:00080000:0.0:1789760769.183465:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973713 is committed 00000100:00100000:0.0:1789760769.183956:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760769.184232:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493803136 00000100:00100000:3.0:1789760769.184242:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493803136 00000100:00100000:3.0:1789760769.184304:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779444126817 time 779478476497 state 0x0 ref 1 00000100:00100000:3.0:1789760769.184320:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779478736913 time 779478476497 state 0x1 ref 1 00000100:00100000:3.0:1789760769.184359:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779478736913 time 779525394807 state 0x0 ref 1 00000100:00100000:3.0:1789760769.184373:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493803136:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760769.184420:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26193920, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc807 00010000:00010000:3.0:1789760769.184430:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06507 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.184445:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760769.184450:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.184455:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06507 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.184467:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06507 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760769.213419:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26193920 -> 26198016 00010000:00010000:0.0:1789760769.213440:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06507 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.213454:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06507 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.213465:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06507 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.213473:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760769.213478:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06507 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.213488:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06507 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.213507:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06507 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.213515:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06507 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.213609:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493803136:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29232us (29652us total) trans 4294973714 rc 0/0 00000100:00100000:0.0:1789760769.213619:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6425 00000100:00100000:0.0:1789760769.213630:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779478736913 time 779525394807 state 0x2 ref 0 00000001:00080000:1.0:1789760769.213830:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973714 is committed 00000100:00100000:0.0:1789760769.218388:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760769.218478:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493803264 00000100:00100000:1.0:1789760769.218488:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493803264 00000100:00100000:1.0:1789760769.218544:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779478736913 time 779525394807 state 0x0 ref 1 00000100:00100000:1.0:1789760769.218559:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779525655223 time 779525394807 state 0x1 ref 1 00000100:00100000:1.0:1789760769.218624:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779525655223 time 779559659507 state 0x0 ref 1 00000100:00100000:1.0:1789760769.218637:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493803264:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760769.218682:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26198016, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc807 00010000:00010000:1.0:1789760769.218692:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0650e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.218708:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760769.218713:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.218719:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0650e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.218734:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0650e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760769.248842:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973715 is committed 00002000:00010000:0.0:1789760769.249467:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26198016 -> 26202112 00010000:00010000:0.0:1789760769.249482:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0650e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.249495:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0650e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.249503:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0650e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.249510:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760769.249515:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0650e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.249523:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0650e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.249541:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0650e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.249548:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0650e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.249640:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493803264:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31001us (31317us total) trans 4294973715 rc 0/0 00000100:00100000:0.0:1789760769.249654:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6426 00000100:00100000:0.0:1789760769.249663:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779525655223 time 779559659507 state 0x2 ref 0 00000100:00100000:2.0:1789760769.253560:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760769.253628:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493803392 00000100:00100000:0.0:1789760769.253638:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493803392 00000100:00100000:0.0:1789760769.253684:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779525655223 time 779559659507 state 0x0 ref 1 00000100:00100000:0.0:1789760769.253699:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779559919923 time 779559659507 state 0x1 ref 1 00000100:00100000:0.0:1789760769.253765:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779559919923 time 779594800793 state 0x0 ref 1 00000100:00100000:0.0:1789760769.253777:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493803392:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760769.253821:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26202112, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc807 00010000:00010000:0.0:1789760769.253830:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06515 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.253846:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=779s v=(0 0) 00010000:00010000:0.0:1789760769.253850:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.253856:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06515 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.253871:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06515 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.277552:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973716 is committed 00002000:00010000:0.0:1789760769.277648:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26202112 -> 26206208 00010000:00010000:0.0:1789760769.277666:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06515 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.277683:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06515 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.277693:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06515 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.277700:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760769.277705:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06515 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.277715:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06515 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.277737:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06515 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.277746:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06515 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.277850:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493803392:12345-192.168.203.60@tcp:4:dd.0 Request processed in 24069us (24294us total) trans 4294973716 rc 0/0 00000100:00100000:0.0:1789760769.277861:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6427 00000100:00100000:0.0:1789760769.277872:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779559919923 time 779594800793 state 0x2 ref 0 00000100:00100000:2.0:1789760769.281354:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760769.281954:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493803520 00000100:00100000:3.0:1789760769.282070:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493803520 00000100:00100000:3.0:1789760769.282125:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779559919923 time 779594800793 state 0x0 ref 1 00000100:00100000:3.0:1789760769.282140:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779595061209 time 779594800793 state 0x1 ref 1 00000100:00100000:3.0:1789760769.282192:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779595061209 time 779623227152 state 0x0 ref 1 00000100:00100000:3.0:1789760769.282205:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493803520:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760769.282256:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26206208, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc807 00010000:00010000:3.0:1789760769.282265:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0651c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.282290:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760769.282295:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.282301:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0651c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.282314:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0651c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.320713:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973717 is committed 00002000:00010000:3.0:1789760769.320868:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26206208 -> 26210304 00010000:00010000:3.0:1789760769.320883:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0651c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.320896:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0651c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.320912:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0651c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.320920:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760769.320925:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0651c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.320934:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0651c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.320952:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0651c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.320974:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0651c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.321061:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493803520:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38854us (39714us total) trans 4294973717 rc 0/0 00000100:00100000:3.0:1789760769.321070:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6428 00000100:00100000:3.0:1789760769.321079:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779595061209 time 779623227152 state 0x2 ref 0 00000100:00100000:0.0:1789760769.324795:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760769.324921:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493803648 00000100:00100000:3.0:1789760769.324930:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493803648 00000100:00100000:3.0:1789760769.325012:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779595061209 time 779623227152 state 0x0 ref 1 00000100:00100000:3.0:1789760769.325028:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779623487568 time 779623227152 state 0x1 ref 1 00000100:00100000:3.0:1789760769.325047:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779623487568 time 779666082809 state 0x0 ref 1 00000100:00100000:3.0:1789760769.325058:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493803648:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760769.325102:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26210304, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc807 00010000:00010000:3.0:1789760769.325110:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06523 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.325126:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760769.325131:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.325136:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06523 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.325149:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06523 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760769.360672:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973718 is committed 00002000:00010000:3.0:1789760769.360745:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26210304 -> 26214400 00010000:00010000:3.0:1789760769.360824:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06523 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.360838:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06523 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.360847:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06523 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.360855:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760769.360861:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06523 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.360871:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06523 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.360889:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06523 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.360896:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06523 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.361041:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493803648:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35980us (36250us total) trans 4294973718 rc 0/0 00000100:00100000:3.0:1789760769.361053:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6429 00000100:00100000:3.0:1789760769.361062:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779623487568 time 779666082809 state 0x2 ref 0 00000100:00100000:1.0:1789760769.364891:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760769.365701:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493803776 00000100:00100000:0.0:1789760769.365712:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493803776 00000100:00100000:0.0:1789760769.365773:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779623487568 time 779666082809 state 0x0 ref 1 00000100:00100000:0.0:1789760769.365787:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779666343225 time 779666082809 state 0x1 ref 1 00000100:00100000:0.0:1789760769.365844:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779666343225 time 779706879517 state 0x0 ref 1 00000100:00100000:0.0:1789760769.365856:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493803776:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760769.365902:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26214400, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xc807 00010000:00010000:0.0:1789760769.365910:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0652a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.365926:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=779s v=(0 0) 00010000:00010000:0.0:1789760769.365930:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.365935:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0652a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.365949:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0652a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.402764:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4528380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3303:1876699519257984:0@lo:400:kworker.0 00000100:00100000:1.0:1789760769.402844:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760769.402944:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519257984 00000100:00100000:2.0:1789760769.402952:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519257984 00000100:00100000:2.0:1789760769.403030:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 410 00000100:00100000:2.0:1789760769.403038:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d4ac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3303:x1876699519257984:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760769.405781:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4528e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3304:1876699519258112:0@lo:400:kworker.0 00000100:00100000:0.0:1789760769.405852:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760769.405900:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519258112 00000100:00100000:0.0:1789760769.405907:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519258112 00000100:00100000:0.0:1789760769.405943:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 509 00000100:00100000:0.0:1789760769.405951:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3304:x1876699519258112:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760769.406022:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3304:x1876699519258112:12345-0@lo:400:kworker.0 Request processed in 71us (170us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760769.406030:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 509 00000100:00100000:0.0:1789760769.406095:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4528e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3304:1876699519258112:0@lo:400:kworker.0 00000100:00100000:1.0:1789760769.406870:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4528380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3303:1876699519257984:0@lo:400:kworker.0 00000100:00100000:3.0:1789760769.406969:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760769.406979:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620c87a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3304:1876699519258496:0@lo:400:kworker.0 00000100:00100000:1.0:1789760769.406991:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4528e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3306:1876699519258368:0@lo:400:kworker.0 00000100:00100000:2.0:1789760769.407069:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d4ac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3303:x1876699519257984:12345-0@lo:400:kworker.0 Request processed in 4029us (4222us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760769.407091:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4528380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3305:1876699519258240:0@lo:400:kworker.0 00000100:00100000:2.0:1789760769.407133:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 410 00000100:00100000:0.0:1789760769.407212:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760769.407299:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519258240 00000100:00100000:2.0:1789760769.407306:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519258240 00000100:00100000:2.0:1789760769.407522:0:9460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 510 00000100:00100000:3.0:1789760769.407588:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760769.407595:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760769.407632:0:6686:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519258496 00000100:00100000:3.0:1789760769.407637:0:6686:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519258496 00000100:00100000:2.0:1789760769.407667:0:9460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3305:x1876699519258240:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760769.407720:0:9460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3305:x1876699519258240:12345-0@lo:400:kworker.0 Request processed in 58us (508us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760769.407727:0:9460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 510 00000100:00100000:2.0:1789760769.407737:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519258368 00000100:00100000:2.0:1789760769.407740:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519258368 00000100:00100000:1.0:1789760769.407815:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4528380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3305:1876699519258240:0@lo:400:kworker.0 00000100:00080000:0.0:1789760769.407946:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789760769.407950:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760769.408053:0:6686:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 183 00000100:00100000:3.0:1789760769.408060:0:6686:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c87a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519258496:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760769.408103:0:6686:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c87a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519258496:12345-0@lo:400:kworker.0 Request processed in 43us (515us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760769.408110:0:6686:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 183 00000100:00100000:1.0:1789760769.408126:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620c87a6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3304:1876699519258496:0@lo:400:kworker.0 00000100:00100000:2.0:1789760769.408929:0:9460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 511 00000100:00100000:2.0:1789760769.408938:0:9460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3306:x1876699519258368:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760769.409015:0:9460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3306:x1876699519258368:12345-0@lo:400:kworker.0 Request processed in 78us (1422us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760769.409021:0:9460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 511 00000100:00100000:1.0:1789760769.409537:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4528e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3306:1876699519258368:0@lo:400:kworker.0 00000001:00080000:1.0:1789760769.421243:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973719 is committed 00002000:00010000:0.0:1789760769.421292:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26214400 -> 26218496 00002000:00010000:0.0:1789760769.421298:0:6694:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb blocks from disk: 51207 -> 53255 00010000:00010000:0.0:1789760769.421311:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0652a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.421324:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0652a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.421332:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0652a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.421339:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760769.421344:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0652a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.421430:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0652a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.421454:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0652a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.421462:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e0652a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.421747:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493803776:12345-192.168.203.60@tcp:4:dd.0 Request processed in 55887us (56861us total) trans 4294973719 rc 0/0 00000100:00100000:0.0:1789760769.421768:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6430 00000100:00100000:0.0:1789760769.421779:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779666343225 time 779706879517 state 0x2 ref 0 00000100:00100000:3.0:1789760769.427844:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760769.427922:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493803904 00000100:00100000:0.0:1789760769.427933:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493803904 00000100:00100000:0.0:1789760769.428023:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779666343225 time 779706879517 state 0x0 ref 1 00000100:00100000:0.0:1789760769.428038:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779707139933 time 779706879517 state 0x1 ref 1 00000100:00100000:0.0:1789760769.428066:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779707139933 time 779769101276 state 0x0 ref 1 00000100:00100000:0.0:1789760769.428078:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493803904:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760769.428126:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26218496, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xd007 00010000:00010000:0.0:1789760769.428134:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06531 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.428149:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=779s v=(0 0) 00010000:00010000:0.0:1789760769.428155:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.428160:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06531 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.428173:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06531 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760769.485707:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973720 is committed 00002000:00010000:2.0:1789760769.486498:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26218496 -> 26222592 00010000:00010000:2.0:1789760769.486514:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06531 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.486525:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06531 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.486538:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06531 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.486545:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760769.486549:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06531 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.486557:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06531 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.486574:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06531 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.486583:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06531 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.486679:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493803904:12345-192.168.203.60@tcp:4:dd.0 Request processed in 58598us (58839us total) trans 4294973720 rc 0/0 00000100:00100000:2.0:1789760769.486689:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6431 00000100:00100000:2.0:1789760769.486699:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779707139933 time 779769101276 state 0x2 ref 0 00000100:00100000:3.0:1789760769.497065:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760769.497683:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493804032 00000100:00100000:2.0:1789760769.497693:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493804032 00000100:00100000:2.0:1789760769.497749:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779707139933 time 779769101276 state 0x0 ref 1 00000100:00100000:2.0:1789760769.497764:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779769361692 time 779769101276 state 0x1 ref 1 00000100:00100000:2.0:1789760769.497795:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779769361692 time 779838830745 state 0x0 ref 1 00000100:00100000:2.0:1789760769.497808:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c87a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493804032:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760769.498032:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26222592, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xd007 00010000:00010000:2.0:1789760769.498042:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06538 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.498058:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760769.498062:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.498068:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06538 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.498082:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06538 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760769.584622:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973721 is committed 00002000:00010000:2.0:1789760769.586480:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26222592 -> 26226688 00010000:00010000:2.0:1789760769.586500:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06538 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.586514:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06538 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.586523:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06538 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.586531:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760769.586536:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06538 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.586544:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06538 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.586564:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06538 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.586572:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06538 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.586674:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c87a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493804032:12345-192.168.203.60@tcp:4:dd.0 Request processed in 88864us (89712us total) trans 4294973721 rc 0/0 00000100:00100000:2.0:1789760769.586684:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6432 00000100:00100000:2.0:1789760769.586694:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779769361692 time 779838830745 state 0x2 ref 0 00000100:00100000:3.0:1789760769.589592:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760769.589688:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493804160 00000100:00100000:0.0:1789760769.589698:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493804160 00000100:00100000:0.0:1789760769.589766:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779769361692 time 779838830745 state 0x0 ref 1 00000100:00100000:0.0:1789760769.589783:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779839091161 time 779838830745 state 0x1 ref 1 00000100:00100000:0.0:1789760769.589813:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779839091161 time 779930848704 state 0x0 ref 1 00000100:00100000:0.0:1789760769.589826:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493804160:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760769.589874:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26226688, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xd007 00010000:00010000:0.0:1789760769.589882:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0653f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.589899:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=779s v=(0 0) 00010000:00010000:0.0:1789760769.589904:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.589911:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0653f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.589925:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0653f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.596569:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760769.621957:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26226688 -> 26230784 00002000:00010000:0.0:1789760769.621988:0:13038:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760768 -> 1789760769 00002000:00010000:0.0:1789760769.621991:0:13038:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760768 -> 1789760769 00010000:00010000:0.0:1789760769.622004:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0653f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.622018:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0653f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.622027:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0653f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.622034:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760769.622039:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0653f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.622048:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0653f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.622067:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0653f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.622074:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e0653f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.622173:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493804160:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32344us (32585us total) trans 4294973722 rc 0/0 00000100:00100000:0.0:1789760769.622184:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6433 00000100:00100000:0.0:1789760769.622194:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779839091161 time 779930848704 state 0x2 ref 0 00000001:00080000:1.0:1789760769.624988:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973722 is committed 00000100:00100000:0.0:1789760769.625844:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760769.625924:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493804288 00000100:00100000:1.0:1789760769.625932:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493804288 00000100:00100000:1.0:1789760769.626027:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779839091161 time 779930848704 state 0x0 ref 1 00000100:00100000:1.0:1789760769.626041:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779931109120 time 779930848704 state 0x1 ref 1 00000100:00100000:1.0:1789760769.626100:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779931109120 time 779967135426 state 0x0 ref 1 00000100:00100000:1.0:1789760769.626112:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493804288:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760769.626155:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26230784, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:1.0:1789760769.626164:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06546 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.626179:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760769.626183:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.626189:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06546 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.626202:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06546 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.653549:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973723 is committed 00002000:00010000:2.0:1789760769.653655:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26230784 -> 26234880 00010000:00010000:2.0:1789760769.653671:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06546 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.653685:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06546 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.653695:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06546 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.653709:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760769.653714:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06546 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.653724:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06546 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.653814:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06546 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.653825:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06546 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.654370:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493804288:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28252us (28523us total) trans 4294973723 rc 0/0 00000100:00100000:2.0:1789760769.654383:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6434 00000100:00100000:2.0:1789760769.654393:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779931109120 time 779967135426 state 0x2 ref 0 00000100:00100000:3.0:1789760769.658218:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760769.658518:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493804416 00000100:00100000:1.0:1789760769.658528:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493804416 00000100:00100000:1.0:1789760769.658579:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779931109120 time 779967135426 state 0x0 ref 1 00000100:00100000:1.0:1789760769.658593:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779967395842 time 779967135426 state 0x1 ref 1 00000100:00100000:1.0:1789760769.658619:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779967395842 time 779999655517 state 0x0 ref 1 00000100:00100000:1.0:1789760769.658631:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493804416:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760769.658673:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26234880, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:1.0:1789760769.658682:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0654d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.658697:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760769.658701:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.658707:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0654d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.658720:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0654d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760769.685465:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973724 is committed 00002000:00010000:1.0:1789760769.685553:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26234880 -> 26238976 00010000:00010000:1.0:1789760769.685569:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0654d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.685582:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0654d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.685592:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0654d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.685599:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760769.685604:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0654d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.685614:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0654d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.685633:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0654d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.685641:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0654d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.685768:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493804416:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27133us (27554us total) trans 4294973724 rc 0/0 00000100:00100000:1.0:1789760769.685780:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6435 00000100:00100000:1.0:1789760769.685790:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779967395842 time 779999655517 state 0x2 ref 0 00000100:00100000:1.0:1789760769.690416:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760769.690507:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493804544 00000100:00100000:2.0:1789760769.690516:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493804544 00000100:00100000:2.0:1789760769.690567:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779967395842 time 779999655517 state 0x0 ref 1 00000100:00100000:2.0:1789760769.690582:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779999915933 time 779999655517 state 0x1 ref 1 00000100:00100000:2.0:1789760769.690635:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779999915933 time 780031670053 state 0x0 ref 1 00000100:00100000:2.0:1789760769.690647:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493804544:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760769.690693:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26238976, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:2.0:1789760769.690702:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06554 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.690718:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=780s v=(0 0) 00010000:00010000:2.0:1789760769.690722:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.690728:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06554 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.690742:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06554 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.719585:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973725 is committed 00002000:00010000:1.0:1789760769.719681:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26238976 -> 26243072 00010000:00010000:1.0:1789760769.719696:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06554 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.719718:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06554 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.719728:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06554 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.719741:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760769.719859:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06554 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.719871:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06554 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.719893:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06554 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.719901:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06554 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.720035:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493804544:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29383us (29621us total) trans 4294973725 rc 0/0 00000100:00100000:1.0:1789760769.720045:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6436 00000100:00100000:1.0:1789760769.720054:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779999915933 time 780031670053 state 0x2 ref 0 00000100:00100000:0.0:1789760769.724947:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760769.725347:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493804672 00000100:00100000:3.0:1789760769.725357:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493804672 00000100:00100000:3.0:1789760769.725409:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 779999915933 time 780031670053 state 0x0 ref 1 00000100:00100000:3.0:1789760769.725424:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780031930469 time 780031670053 state 0x1 ref 1 00000100:00100000:3.0:1789760769.725819:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780031930469 time 780066852778 state 0x0 ref 1 00000100:00100000:3.0:1789760769.725834:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493804672:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760769.725882:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26243072, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:3.0:1789760769.725892:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0655b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.725907:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760769.725912:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.725918:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0655b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.725932:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0655b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.773493:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973726 is committed 00002000:00010000:1.0:1789760769.773582:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26243072 -> 26247168 00010000:00010000:1.0:1789760769.773597:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0655b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.773610:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0655b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.773626:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0655b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.773634:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760769.773638:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0655b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.773647:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0655b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.773666:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0655b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.773674:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0655b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.773774:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493804672:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47937us (48830us total) trans 4294973726 rc 0/0 00000100:00100000:1.0:1789760769.773784:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6437 00000100:00100000:1.0:1789760769.773796:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780031930469 time 780066852778 state 0x2 ref 0 00000100:00100000:3.0:1789760769.776732:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760769.777026:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493804800 00000100:00100000:1.0:1789760769.777038:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493804800 00000100:00100000:1.0:1789760769.777194:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780031930469 time 780066852778 state 0x0 ref 1 00000100:00100000:1.0:1789760769.777209:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780067113194 time 780066852778 state 0x1 ref 1 00000100:00100000:1.0:1789760769.777240:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780067113194 time 780118275956 state 0x0 ref 1 00000100:00100000:1.0:1789760769.777252:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d08e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493804800:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760769.777310:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26247168, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:1.0:1789760769.777327:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06562 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.777343:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760769.777347:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.777354:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06562 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.777368:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06562 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760769.849860:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973727 is committed 00002000:00010000:2.0:1789760769.854209:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26247168 -> 26251264 00010000:00010000:2.0:1789760769.854233:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06562 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.854247:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06562 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.854256:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06562 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.854275:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760769.854280:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06562 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.854289:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06562 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.854310:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06562 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.854812:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06562 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.855041:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d08e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493804800:12345-192.168.203.60@tcp:4:dd.0 Request processed in 77785us (78313us total) trans 4294973727 rc 0/0 00000100:00100000:2.0:1789760769.855053:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6438 00000100:00100000:2.0:1789760769.855063:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780067113194 time 780118275956 state 0x2 ref 0 00000100:00100000:3.0:1789760769.865175:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760769.865295:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493804928 00000100:00100000:0.0:1789760769.865306:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493804928 00000100:00100000:0.0:1789760769.865416:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780067113194 time 780118275956 state 0x0 ref 1 00000100:00100000:0.0:1789760769.865432:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780118536372 time 780118275956 state 0x1 ref 1 00000100:00100000:0.0:1789760769.865467:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780118536372 time 780206502834 state 0x0 ref 1 00000100:00100000:0.0:1789760769.865480:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493804928:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760769.865533:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26251264, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:0.0:1789760769.865542:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06569 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.865559:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=780s v=(0 0) 00010000:00010000:0.0:1789760769.865564:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.865571:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06569 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.865591:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06569 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.914140:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc1d0a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3304:1876699519258624:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760769.914231:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000001:00080000:1.0:1789760769.917640:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973728 is committed 00000100:00100000:3.0:1789760769.921707:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519258624 00000100:00100000:3.0:1789760769.921720:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519258624 00002000:00010000:2.0:1789760769.921765:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26251264 -> 26255360 00010000:00010000:2.0:1789760769.921785:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06569 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.921798:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06569 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.921808:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06569 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.921815:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760769.921820:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06569 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.921829:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06569 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.921846:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06569 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.921853:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06569 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.921949:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493804928:12345-192.168.203.60@tcp:4:dd.0 Request processed in 56468us (56780us total) trans 4294973728 rc 0/0 00000100:00100000:0.0:1789760769.922073:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 211 00000100:00100000:0.0:1789760769.922083:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519258624:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760769.922116:0:6690:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760769.922177:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519258624:12345-0@lo:13:osp-pre-1-0.0 Request processed in 95us (7945us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760769.922184:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 211 00000100:00100000:0.0:1789760769.922237:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc1d0a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3304:1876699519258624:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760769.923206:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6439 00000100:00100000:2.0:1789760769.923219:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780118536372 time 780206502834 state 0x2 ref 0 00000100:00100000:0.0:1789760769.932166:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760769.932296:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493805056 00000100:00100000:2.0:1789760769.932311:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493805056 00000100:00100000:2.0:1789760769.932381:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780118536372 time 780206502834 state 0x0 ref 1 00000100:00100000:2.0:1789760769.932397:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780206763250 time 780206502834 state 0x1 ref 1 00000100:00100000:2.0:1789760769.932438:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780206763250 time 780273472787 state 0x0 ref 1 00000100:00100000:2.0:1789760769.932452:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493805056:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760769.932507:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26255360, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:2.0:1789760769.932519:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06570 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.932535:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=780s v=(0 0) 00010000:00010000:2.0:1789760769.932540:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.932547:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06570 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.932561:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06570 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760769.974598:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26255360 -> 26259456 00010000:00010000:2.0:1789760769.974621:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06570 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.974634:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06570 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.974645:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06570 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.974653:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760769.974658:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06570 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.974667:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06570 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.974694:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06570 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.974702:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06570 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.974944:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493805056:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42489us (42787us total) trans 4294973729 rc 0/0 00000100:00100000:2.0:1789760769.974956:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6440 00000100:00100000:2.0:1789760769.974998:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780206763250 time 780273472787 state 0x2 ref 0 00000001:00080000:3.0:1789760769.975075:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973729 is committed 00000100:00100000:3.0:1789760769.979739:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760769.979818:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493805184 00000100:00100000:1.0:1789760769.979828:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493805184 00000100:00100000:1.0:1789760769.979885:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780206763250 time 780273472787 state 0x0 ref 1 00000100:00100000:1.0:1789760769.979900:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780273733203 time 780273472787 state 0x1 ref 1 00000100:00100000:1.0:1789760769.979928:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780273733203 time 780320963240 state 0x0 ref 1 00000100:00100000:1.0:1789760769.979944:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493805184:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760769.980027:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26259456, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:1.0:1789760769.980037:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06577 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.980054:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760769.980058:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.980065:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06577 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.980079:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06577 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.041400:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973730 is committed 00002000:00010000:1.0:1789760770.041499:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26259456 -> 26263552 00010000:00010000:1.0:1789760770.041515:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06577 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.041529:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06577 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.041538:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06577 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.041546:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760770.041551:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06577 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.041561:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06577 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.041582:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06577 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.041590:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06577 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.041690:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493805184:12345-192.168.203.60@tcp:4:dd.0 Request processed in 61743us (61956us total) trans 4294973730 rc 0/0 00000100:00100000:1.0:1789760770.041701:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6441 00000100:00100000:1.0:1789760770.041710:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780273733203 time 780320963240 state 0x2 ref 0 00000100:00100000:1.0:1789760770.045582:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760770.045673:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493805312 00000100:00100000:0.0:1789760770.045685:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493805312 00000100:00100000:0.0:1789760770.045810:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780273733203 time 780320963240 state 0x0 ref 1 00000100:00100000:0.0:1789760770.045827:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780321223656 time 780320963240 state 0x1 ref 1 00000100:00100000:0.0:1789760770.045858:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780321223656 time 780386893828 state 0x0 ref 1 00000100:00100000:0.0:1789760770.045873:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493805312:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760770.045920:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26263552, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:0.0:1789760770.045929:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0657e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.045950:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=780s v=(0 0) 00010000:00010000:0.0:1789760770.045956:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.046248:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0657e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.046274:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0657e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760770.113399:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26263552 -> 26267648 00010000:00010000:0.0:1789760770.113462:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0657e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.113476:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0657e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.113485:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0657e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.113492:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760770.113496:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0657e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.113506:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0657e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.113525:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0657e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.113534:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0657e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.113618:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493805312:12345-192.168.203.60@tcp:4:dd.0 Request processed in 67743us (68040us total) trans 4294973731 rc 0/0 00000100:00100000:0.0:1789760770.113628:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6442 00000100:00100000:0.0:1789760770.113638:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780321223656 time 780386893828 state 0x2 ref 0 00000001:00080000:1.0:1789760770.113897:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973731 is committed 00000100:00100000:3.0:1789760770.116847:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760770.117012:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493805440 00000100:00100000:2.0:1789760770.117023:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493805440 00000100:00100000:2.0:1789760770.117078:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780321223656 time 780386893828 state 0x0 ref 1 00000100:00100000:2.0:1789760770.117095:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780387154244 time 780386893828 state 0x1 ref 1 00000100:00100000:2.0:1789760770.117131:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780387154244 time 780458166658 state 0x0 ref 1 00000100:00100000:2.0:1789760770.117143:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493805440:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760770.117189:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26267648, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:2.0:1789760770.117198:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06585 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.117218:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=780s v=(0 0) 00010000:00010000:2.0:1789760770.117223:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.117229:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06585 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.117243:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06585 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760770.150729:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973732 is committed 00002000:00010000:0.0:1789760770.150841:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26267648 -> 26271744 00010000:00010000:0.0:1789760770.150856:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06585 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.150868:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06585 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.150876:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06585 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.150883:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760770.150888:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06585 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.150897:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06585 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.150922:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06585 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.150930:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06585 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.151155:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493805440:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34010us (34316us total) trans 4294973732 rc 0/0 00000100:00100000:1.0:1789760770.151164:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6443 00000100:00100000:1.0:1789760770.151172:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780387154244 time 780458166658 state 0x2 ref 0 00000100:00100000:3.0:1789760770.154774:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760770.154953:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493805568 00000100:00100000:2.0:1789760770.155081:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493805568 00000100:00100000:2.0:1789760770.155140:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780387154244 time 780458166658 state 0x0 ref 1 00000100:00100000:2.0:1789760770.155157:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780458427074 time 780458166658 state 0x1 ref 1 00000100:00100000:2.0:1789760770.155220:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780458427074 time 780496255033 state 0x0 ref 1 00000100:00100000:2.0:1789760770.155234:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d0a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493805568:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760770.155307:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26271744, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:2.0:1789760770.155317:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0658c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.155340:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=780s v=(0 0) 00010000:00010000:2.0:1789760770.155345:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.155359:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0658c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.155389:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0658c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760770.207664:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26271744 -> 26275840 00010000:00010000:1.0:1789760770.207685:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0658c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.207703:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0658c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.207712:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0658c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.207718:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760770.207723:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0658c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.207731:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0658c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.207761:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0658c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.207769:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e0658c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.207864:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d0a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493805568:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52629us (53097us total) trans 4294973733 rc 0/0 00000100:00100000:1.0:1789760770.207874:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6444 00000100:00100000:1.0:1789760770.207883:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780458427074 time 780496255033 state 0x2 ref 0 00000001:00080000:1.0:1789760770.207920:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973733 is committed 00000100:00100000:0.0:1789760770.229712:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760770.229934:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493805696 00000100:00100000:1.0:1789760770.229945:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493805696 00000100:00100000:1.0:1789760770.230034:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780458427074 time 780496255033 state 0x0 ref 1 00000100:00100000:1.0:1789760770.230050:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780496515449 time 780496255033 state 0x1 ref 1 00000100:00100000:1.0:1789760770.230078:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780496515449 time 780571113882 state 0x0 ref 1 00000100:00100000:1.0:1789760770.230093:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493805696:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760770.230144:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26275840, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:1.0:1789760770.230153:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06593 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.230169:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760770.230173:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.230179:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06593 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.230193:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06593 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760770.315831:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973734 is committed 00002000:00010000:0.0:1789760770.315907:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26275840 -> 26279936 00010000:00010000:0.0:1789760770.315922:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06593 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.315936:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06593 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.315945:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06593 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.315953:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760770.315957:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06593 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.315996:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06593 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.316016:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06593 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.316024:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06593 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.316112:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493805696:12345-192.168.203.60@tcp:4:dd.0 Request processed in 86017us (86408us total) trans 4294973734 rc 0/0 00000100:00100000:0.0:1789760770.316123:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6445 00000100:00100000:0.0:1789760770.316132:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780496515449 time 780571113882 state 0x2 ref 0 00000100:00100000:3.0:1789760770.321893:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760770.322147:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493805824 00000100:00100000:0.0:1789760770.322156:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493805824 00000100:00100000:0.0:1789760770.322210:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780496515449 time 780571113882 state 0x0 ref 1 00000100:00100000:0.0:1789760770.322225:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780571374298 time 780571113882 state 0x1 ref 1 00000100:00100000:0.0:1789760770.322259:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780571374298 time 780663294828 state 0x0 ref 1 00000100:00100000:0.0:1789760770.322281:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493805824:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760770.322336:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26279936, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:0.0:1789760770.322345:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0659a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.323071:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=780s v=(0 0) 00010000:00010000:0.0:1789760770.323076:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.323083:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0659a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.323105:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0659a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760770.355657:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26279936 -> 26284032 00010000:00010000:2.0:1789760770.355677:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0659a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.355690:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0659a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.355700:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0659a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.355708:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760770.355713:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0659a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.355722:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0659a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.355744:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0659a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.355751:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e0659a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.355850:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493805824:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33566us (33963us total) trans 4294973735 rc 0/0 00000100:00100000:2.0:1789760770.355860:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6446 00000100:00100000:2.0:1789760770.355870:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780571374298 time 780663294828 state 0x2 ref 0 00000100:00100000:3.0:1789760770.360456:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760770.360535:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493805952 00000100:00100000:2.0:1789760770.360545:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493805952 00000100:00100000:2.0:1789760770.360599:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780571374298 time 780663294828 state 0x0 ref 1 00000100:00100000:2.0:1789760770.360615:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780663555244 time 780663294828 state 0x1 ref 1 00000100:00100000:2.0:1789760770.360647:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780663555244 time 780701682689 state 0x0 ref 1 00000100:00100000:2.0:1789760770.360660:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493805952:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760770.360709:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26284032, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:2.0:1789760770.360719:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e065a1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.360735:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=780s v=(0 0) 00010000:00010000:2.0:1789760770.360740:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.360759:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e065a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.360775:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e065a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760770.362089:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973735 is committed 00000001:00080000:1.0:1789760770.400209:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973736 is committed 00002000:00010000:3.0:1789760770.400237:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26284032 -> 26288128 00010000:00010000:3.0:1789760770.400253:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e065a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.400279:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e065a1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.400288:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e065a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.400295:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760770.400300:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e065a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.400309:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e065a1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.400329:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e065a1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.400337:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e065a1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.400957:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493805952:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40294us (40506us total) trans 4294973736 rc 0/0 00000100:00100000:3.0:1789760770.401017:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6447 00000100:00100000:3.0:1789760770.401028:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780663555244 time 780701682689 state 0x2 ref 0 00000100:00100000:0.0:1789760770.404757:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760770.404851:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493806080 00000100:00100000:1.0:1789760770.404861:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493806080 00000100:00100000:1.0:1789760770.404912:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780663555244 time 780701682689 state 0x0 ref 1 00000100:00100000:1.0:1789760770.404931:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780701943105 time 780701682689 state 0x1 ref 1 00000100:00100000:2.0:1789760770.405078:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780701943105 time 780746110881 state 0x0 ref 1 00000100:00100000:2.0:1789760770.405095:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493806080:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760770.405150:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26288128, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:2.0:1789760770.405159:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e065a8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.405174:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=780s v=(0 0) 00010000:00010000:2.0:1789760770.405179:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.405185:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e065a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.405198:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e065a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760770.480633:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973737 is committed 00002000:00010000:0.0:1789760770.480776:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26288128 -> 26292224 00010000:00010000:0.0:1789760770.480796:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e065a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.480807:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e065a8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.480816:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e065a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.480823:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760770.480827:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e065a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.480836:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e065a8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.480855:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e065a8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.480862:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e065a8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.481006:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493806080:12345-192.168.203.60@tcp:4:dd.0 Request processed in 75910us (76254us total) trans 4294973737 rc 0/0 00000100:00100000:0.0:1789760770.481016:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6448 00000100:00100000:0.0:1789760770.481234:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780701943105 time 780746110881 state 0x2 ref 0 00000100:00100000:0.0:1789760770.486382:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760770.486439:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493806208 00000100:00100000:3.0:1789760770.486449:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493806208 00000100:00100000:3.0:1789760770.486501:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780701943105 time 780746110881 state 0x0 ref 1 00000100:00100000:3.0:1789760770.486516:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780746371297 time 780746110881 state 0x1 ref 1 00000100:00100000:3.0:1789760770.486547:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780746371297 time 780827582382 state 0x0 ref 1 00000100:00100000:3.0:1789760770.486559:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493806208:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760770.486605:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26292224, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:3.0:1789760770.486614:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e065af lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.486629:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760770.486633:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.486639:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e065af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.486657:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e065af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.536187:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973738 is committed 00002000:00010000:1.0:1789760770.536303:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26292224 -> 26296320 00010000:00010000:1.0:1789760770.536330:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e065af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.536344:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e065af lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.536352:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e065af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.536359:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760770.536363:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e065af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.536372:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e065af lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.536391:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e065af lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.536399:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e065af lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.536673:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493806208:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50108us (50295us total) trans 4294973738 rc 0/0 00000100:00100000:1.0:1789760770.536686:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6449 00000100:00100000:1.0:1789760770.536695:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780746371297 time 780827582382 state 0x2 ref 0 00000100:00100000:2.0:1789760770.540234:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760770.540347:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493806336 00000100:00100000:1.0:1789760770.540357:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493806336 00000100:00100000:1.0:1789760770.540414:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780746371297 time 780827582382 state 0x0 ref 1 00000100:00100000:1.0:1789760770.540599:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780827842798 time 780827582382 state 0x1 ref 1 00000100:00100000:1.0:1789760770.540633:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780827842798 time 780881668952 state 0x0 ref 1 00000100:00100000:1.0:1789760770.540647:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493806336:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760770.540705:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26296320, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:1.0:1789760770.540717:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e065b6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.540733:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760770.540738:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.540744:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e065b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.540761:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e065b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.599368:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973739 is committed 00002000:00010000:0.0:1789760770.600285:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26296320 -> 26300416 00010000:00010000:0.0:1789760770.600309:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e065b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.600323:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e065b6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.600332:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e065b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.600340:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760770.600344:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e065b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.600353:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e065b6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.600374:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e065b6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.600382:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e065b6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.600630:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493806336:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59980us (60399us total) trans 4294973739 rc 0/0 00000100:00100000:2.0:1789760770.600643:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6450 00000100:00100000:2.0:1789760770.600653:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780827842798 time 780881668952 state 0x2 ref 0 00000100:00100000:0.0:1789760770.606687:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760770.607002:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493806464 00000100:00100000:2.0:1789760770.607012:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493806464 00000100:00100000:2.0:1789760770.607076:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780827842798 time 780881668952 state 0x0 ref 1 00000100:00100000:2.0:1789760770.607095:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780881929368 time 780881668952 state 0x1 ref 1 00000100:00100000:2.0:1789760770.607119:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780881929368 time 780948155478 state 0x0 ref 1 00000100:00100000:2.0:1789760770.607132:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc352c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493806464:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760770.607179:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26300416, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xd007 00010000:00010000:2.0:1789760770.607187:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e065bd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.607202:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=780s v=(0 0) 00010000:00010000:2.0:1789760770.607207:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.607212:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e065bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.607226:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e065bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.618134:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760770.644624:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26300416 -> 26304512 00002000:00010000:2.0:1789760770.644633:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760769 -> 1789760770 00002000:00010000:2.0:1789760770.644636:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760769 -> 1789760770 00010000:00010000:2.0:1789760770.644650:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e065bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.644663:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e065bd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.644678:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e065bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.644685:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760770.644690:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e065bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.644698:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e065bd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.644715:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e065bd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.644722:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e065bd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.644803:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493806464:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37669us (38122us total) trans 4294973740 rc 0/0 00000100:00100000:2.0:1789760770.644813:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6451 00000100:00100000:2.0:1789760770.644822:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780881929368 time 780948155478 state 0x2 ref 0 00000001:00080000:0.0:1789760770.645191:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973740 is committed 00000100:00100000:0.0:1789760770.649824:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760770.650015:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493806592 00000100:00100000:2.0:1789760770.650026:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493806592 00000100:00100000:2.0:1789760770.650222:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780881929368 time 780948155478 state 0x0 ref 1 00000100:00100000:2.0:1789760770.650238:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780948415894 time 780948155478 state 0x1 ref 1 00000100:00100000:2.0:1789760770.650285:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780948415894 time 780991320697 state 0x0 ref 1 00000100:00100000:2.0:1789760770.650299:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493806592:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760770.650471:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26304512, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:2.0:1789760770.650482:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e065c4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.650496:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=780s v=(0 0) 00010000:00010000:2.0:1789760770.650501:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.650508:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e065c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.650521:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e065c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760770.700480:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26304512 -> 26308608 00010000:00010000:3.0:1789760770.700498:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e065c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.700511:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e065c4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.700520:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e065c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.700527:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760770.700532:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e065c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.700542:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e065c4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.700562:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e065c4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.700569:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e065c4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.700654:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493806592:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50354us (50835us total) trans 4294973741 rc 0/0 00000100:00100000:3.0:1789760770.700663:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6452 00000100:00100000:3.0:1789760770.700672:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780948415894 time 780991320697 state 0x2 ref 0 00000001:00080000:1.0:1789760770.700843:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973741 is committed 00000100:00100000:2.0:1789760770.704183:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760770.704354:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493806720 00000100:00100000:1.0:1789760770.704364:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493806720 00000100:00100000:1.0:1789760770.704419:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780948415894 time 780991320697 state 0x0 ref 1 00000100:00100000:1.0:1789760770.704432:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780991581113 time 780991320697 state 0x1 ref 1 00000100:00100000:1.0:1789760770.704456:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780991581113 time 781045491596 state 0x0 ref 1 00000100:00100000:1.0:1789760770.704468:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493806720:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760770.704515:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26308608, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:1.0:1789760770.704524:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e065cb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.704540:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=781s v=(0 0) 00010000:00010000:1.0:1789760770.704545:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.704551:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e065cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.704564:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e065cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760770.749724:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26308608 -> 26312704 00010000:00010000:2.0:1789760770.749743:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e065cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.749757:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e065cb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.749766:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e065cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760770.749831:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973742 is committed 00010000:00010000:2.0:1789760770.749845:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760770.749850:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e065cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.749861:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e065cb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.749882:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e065cb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.749890:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e065cb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.750016:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493806720:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45544us (45838us total) trans 4294973742 rc 0/0 00000100:00100000:2.0:1789760770.750089:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6453 00000100:00100000:2.0:1789760770.750099:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780991581113 time 781045491596 state 0x2 ref 0 00000100:00100000:2.0:1789760770.754305:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760770.754409:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493806848 00000100:00100000:1.0:1789760770.754420:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493806848 00000100:00100000:1.0:1789760770.754484:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 780991581113 time 781045491596 state 0x0 ref 1 00000100:00100000:1.0:1789760770.754501:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781045752012 time 781045491596 state 0x1 ref 1 00000100:00100000:0.0:1789760770.754584:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781045752012 time 781095617183 state 0x0 ref 1 00000100:00100000:0.0:1789760770.754600:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493806848:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760770.754655:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26312704, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:0.0:1789760770.754664:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e065d2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.754680:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=781s v=(0 0) 00010000:00010000:0.0:1789760770.754685:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.754692:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e065d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.754706:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e065d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.783504:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973743 is committed 00002000:00010000:3.0:1789760770.783533:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26312704 -> 26316800 00010000:00010000:3.0:1789760770.783548:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e065d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.783561:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e065d2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.783571:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e065d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.783580:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760770.783586:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e065d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.783596:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e065d2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.783617:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e065d2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.783631:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e065d2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.783724:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493806848:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29123us (29424us total) trans 4294973743 rc 0/0 00000100:00100000:3.0:1789760770.783733:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6454 00000100:00100000:3.0:1789760770.783743:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781045752012 time 781095617183 state 0x2 ref 0 00000100:00100000:0.0:1789760770.787122:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760770.787231:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493806976 00000100:00100000:2.0:1789760770.787241:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493806976 00000100:00100000:2.0:1789760770.787308:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781045752012 time 781095617183 state 0x0 ref 1 00000100:00100000:2.0:1789760770.787324:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781095877599 time 781095617183 state 0x1 ref 1 00000100:00100000:2.0:1789760770.787360:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781095877599 time 781128395460 state 0x0 ref 1 00000100:00100000:2.0:1789760770.787373:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493806976:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760770.787427:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26316800, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:2.0:1789760770.787436:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e065d9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.787452:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=781s v=(0 0) 00010000:00010000:2.0:1789760770.787456:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.787463:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e065d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.787477:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e065d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760770.819164:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26316800 -> 26320896 00010000:00010000:2.0:1789760770.819183:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e065d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.819197:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e065d9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.819206:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e065d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.819213:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760770.819219:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e065d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.819228:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e065d9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.819247:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e065d9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.819255:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e065d9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.819363:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493806976:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31988us (32245us total) trans 4294973744 rc 0/0 00000100:00100000:2.0:1789760770.819373:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6455 00000100:00100000:2.0:1789760770.819383:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781095877599 time 781128395460 state 0x2 ref 0 00000001:00080000:3.0:1789760770.820025:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973744 is committed 00000100:00100000:2.0:1789760770.830070:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760770.830151:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493807104 00000100:00100000:3.0:1789760770.830162:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493807104 00000100:00100000:3.0:1789760770.830216:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781095877599 time 781128395460 state 0x0 ref 1 00000100:00100000:3.0:1789760770.830230:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781128655876 time 781128395460 state 0x1 ref 1 00000100:00100000:3.0:1789760770.830247:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781128655876 time 781171283535 state 0x0 ref 1 00000100:00100000:3.0:1789760770.830273:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c3094700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493807104:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760770.830328:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26320896, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:3.0:1789760770.830337:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e065e0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.830353:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760770.830358:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.830365:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e065e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.830379:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e065e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.856509:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973745 is committed 00002000:00010000:2.0:1789760770.856643:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26320896 -> 26324992 00010000:00010000:2.0:1789760770.856658:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e065e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.856671:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e065e0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.856680:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e065e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.856693:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760770.856698:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e065e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.856708:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e065e0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.856726:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e065e0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.856734:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e065e0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.856822:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c3094700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493807104:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26560us (26757us total) trans 4294973745 rc 0/0 00000100:00100000:2.0:1789760770.856832:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6456 00000100:00100000:2.0:1789760770.856841:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781128655876 time 781171283535 state 0x2 ref 0 00000100:00100000:1.0:1789760770.863329:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760770.863393:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493807232 00000100:00100000:2.0:1789760770.863403:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493807232 00000100:00100000:2.0:1789760770.863459:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781128655876 time 781171283535 state 0x0 ref 1 00000100:00100000:2.0:1789760770.863474:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781171543951 time 781171283535 state 0x1 ref 1 00000100:00100000:2.0:1789760770.863520:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781171543951 time 781204547268 state 0x0 ref 1 00000100:00100000:2.0:1789760770.863536:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493807232:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760770.863586:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26324992, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:2.0:1789760770.863595:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e065e7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.863616:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=781s v=(0 0) 00010000:00010000:2.0:1789760770.863620:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.863627:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e065e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.863641:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e065e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760770.899857:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26324992 -> 26329088 00010000:00010000:2.0:1789760770.899875:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e065e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.899887:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e065e7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.899895:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e065e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.899902:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760770.899907:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e065e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.899917:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e065e7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.899935:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e065e7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.899943:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e065e7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.900046:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493807232:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36508us (36731us total) trans 4294973746 rc 0/0 00000100:00100000:2.0:1789760770.900056:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6457 00000100:00100000:2.0:1789760770.900066:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781171543951 time 781204547268 state 0x2 ref 0 00000001:00080000:0.0:1789760770.900191:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973746 is committed 00000100:00100000:0.0:1789760770.903853:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760770.903915:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493807360 00000100:00100000:3.0:1789760770.903926:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493807360 00000100:00100000:3.0:1789760770.904251:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781171543951 time 781204547268 state 0x0 ref 1 00000100:00100000:3.0:1789760770.904279:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781204807684 time 781204547268 state 0x1 ref 1 00000100:00100000:3.0:1789760770.904313:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781204807684 time 781245347946 state 0x0 ref 1 00000100:00100000:3.0:1789760770.904326:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493807360:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760770.904377:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26329088, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:3.0:1789760770.904387:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e065ee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.904403:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760770.904407:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.904413:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e065ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.904427:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e065ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760770.946167:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26329088 -> 26333184 00010000:00010000:3.0:1789760770.946186:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e065ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.946199:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e065ee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.946208:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e065ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.946216:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760770.946221:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e065ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.946231:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e065ee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.946250:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e065ee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.946266:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e065ee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.946370:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493807360:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42042us (42521us total) trans 4294973747 rc 0/0 00000100:00100000:3.0:1789760770.946384:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6458 00000100:00100000:3.0:1789760770.946394:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781204807684 time 781245347946 state 0x2 ref 0 00000001:00080000:1.0:1789760770.946949:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973747 is committed 00000100:00100000:3.0:1789760770.956239:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760770.956312:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493807488 00000100:00100000:0.0:1789760770.956322:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493807488 00000100:00100000:0.0:1789760770.956443:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781204807684 time 781245347946 state 0x0 ref 1 00000100:00100000:0.0:1789760770.956459:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781245608362 time 781245347946 state 0x1 ref 1 00000100:00100000:0.0:1789760770.956539:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781245608362 time 781297573417 state 0x0 ref 1 00000100:00100000:0.0:1789760770.956552:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb452a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493807488:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760770.956600:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26333184, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:0.0:1789760770.956609:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e065f5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.956624:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=781s v=(0 0) 00010000:00010000:0.0:1789760770.956629:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.956636:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e065f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.956650:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e065f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760770.993306:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26333184 -> 26337280 00010000:00010000:0.0:1789760770.993540:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e065f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.993555:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e065f5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.993570:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e065f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.993578:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760770.993583:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e065f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.993593:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e065f5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.993614:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e065f5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.993623:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e065f5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.993736:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb452a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493807488:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37180us (37499us total) trans 4294973748 rc 0/0 00000100:00100000:0.0:1789760770.993747:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6459 00000100:00100000:0.0:1789760770.993758:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781245608362 time 781297573417 state 0x2 ref 0 00000001:00080000:2.0:1789760770.993851:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973748 is committed 00000100:00100000:0.0:1789760771.018534:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760771.021499:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493807616 00000100:00100000:2.0:1789760771.021514:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493807616 00000100:00100000:0.0:1789760771.021745:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781245608362 time 781297573417 state 0x0 ref 1 00000100:00100000:0.0:1789760771.021761:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781297833833 time 781297573417 state 0x1 ref 1 00000100:00100000:0.0:1789760771.021794:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781297833833 time 781362830447 state 0x0 ref 1 00000100:00100000:0.0:1789760771.021807:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbedf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493807616:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760771.021855:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26337280, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:0.0:1789760771.021864:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e065fc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.021879:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=781s v=(0 0) 00010000:00010000:0.0:1789760771.021883:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.021889:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e065fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.021903:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e065fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760771.023178:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760771.057941:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973749 is committed 00002000:00010000:1.0:1789760771.058933:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26337280 -> 26341376 00010000:00010000:1.0:1789760771.058948:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e065fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.059044:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e065fc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.059055:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e065fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.059063:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760771.059069:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e065fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.059078:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e065fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.059098:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e065fc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.059106:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e065fc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.059232:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbedf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493807616:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37422us (40703us total) trans 4294973749 rc 0/0 00000100:00100000:1.0:1789760771.059246:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6460 00000100:00100000:1.0:1789760771.059266:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781297833833 time 781362830447 state 0x2 ref 0 00000100:00100000:1.0:1789760771.062881:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760771.063109:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493807744 00000100:00100000:2.0:1789760771.063120:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493807744 00000100:00100000:3.0:1789760771.063254:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781297833833 time 781362830447 state 0x0 ref 1 00000100:00100000:3.0:1789760771.063281:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781363090863 time 781362830447 state 0x1 ref 1 00000100:00100000:3.0:1789760771.063458:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781363090863 time 781404491925 state 0x0 ref 1 00000100:00100000:3.0:1789760771.063472:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493807744:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760771.063522:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26341376, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:3.0:1789760771.063532:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06603 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.063548:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760771.063552:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.063558:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06603 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.063572:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06603 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.096038:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973750 is committed 00002000:00010000:2.0:1789760771.096132:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26341376 -> 26345472 00010000:00010000:2.0:1789760771.096146:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06603 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.096159:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06603 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.096173:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06603 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.096180:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760771.096184:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06603 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.096192:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06603 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.096211:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06603 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.096219:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06603 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.096329:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493807744:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32855us (33454us total) trans 4294973750 rc 0/0 00000100:00100000:2.0:1789760771.096340:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6461 00000100:00100000:2.0:1789760771.096795:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781363090863 time 781404491925 state 0x2 ref 0 00000100:00100000:3.0:1789760771.101723:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760771.101827:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493807872 00000100:00100000:0.0:1789760771.101837:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493807872 00000100:00100000:0.0:1789760771.101891:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781363090863 time 781404491925 state 0x0 ref 1 00000100:00100000:0.0:1789760771.101905:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781404752341 time 781404491925 state 0x1 ref 1 00000100:00100000:0.0:1789760771.101930:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781404752341 time 781442966335 state 0x0 ref 1 00000100:00100000:0.0:1789760771.101941:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb453800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493807872:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760771.102029:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26345472, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:0.0:1789760771.102038:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0660a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.102053:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=781s v=(0 0) 00010000:00010000:0.0:1789760771.102057:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.102063:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0660a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.102076:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0660a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760771.145939:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26345472 -> 26349568 00010000:00010000:2.0:1789760771.145978:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0660a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.145997:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0660a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.146006:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0660a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.146014:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760771.146020:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0660a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.146029:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0660a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.146049:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0660a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.146056:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0660a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.146146:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb453800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493807872:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44202us (44427us total) trans 4294973751 rc 0/0 00000100:00100000:2.0:1789760771.146158:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6462 00000100:00100000:2.0:1789760771.146168:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781404752341 time 781442966335 state 0x2 ref 0 00000001:00080000:2.0:1789760771.147291:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973751 is committed 00000100:00100000:0.0:1789760771.149531:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760771.151449:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493808000 00000100:00100000:2.0:1789760771.151463:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493808000 00000100:00100000:2.0:1789760771.151524:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781404752341 time 781442966335 state 0x0 ref 1 00000100:00100000:2.0:1789760771.151541:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781443226751 time 781442966335 state 0x1 ref 1 00000100:00100000:2.0:1789760771.151616:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781443226751 time 781492650704 state 0x0 ref 1 00000100:00100000:2.0:1789760771.151632:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493808000:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760771.151680:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26349568, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:2.0:1789760771.151689:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06611 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.151705:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=781s v=(0 0) 00010000:00010000:2.0:1789760771.151710:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.151717:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06611 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.151731:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06611 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760771.186848:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973752 is committed 00002000:00010000:2.0:1789760771.187018:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26349568 -> 26353664 00010000:00010000:2.0:1789760771.187034:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06611 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.187214:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06611 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.187230:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06611 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.187237:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760771.187242:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06611 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.187252:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06611 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.187283:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06611 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.187291:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06611 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.187390:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493808000:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35756us (37861us total) trans 4294973752 rc 0/0 00000100:00100000:2.0:1789760771.187400:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6463 00000100:00100000:2.0:1789760771.187409:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781443226751 time 781492650704 state 0x2 ref 0 00000100:00100000:2.0:1789760771.199709:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760771.199836:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493808128 00000100:00100000:1.0:1789760771.199846:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493808128 00000100:00100000:1.0:1789760771.199898:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781443226751 time 781492650704 state 0x0 ref 1 00000100:00100000:1.0:1789760771.199914:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781492911120 time 781492650704 state 0x1 ref 1 00000100:00100000:1.0:1789760771.200031:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781492911120 time 781541065559 state 0x0 ref 1 00000100:00100000:1.0:1789760771.200045:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493808128:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760771.200158:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26353664, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:1.0:1789760771.200167:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06618 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.200183:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=781s v=(0 0) 00010000:00010000:1.0:1789760771.200187:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.200195:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06618 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.200208:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06618 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.246654:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973753 is committed 00002000:00010000:2.0:1789760771.246802:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26353664 -> 26357760 00010000:00010000:2.0:1789760771.246817:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06618 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.246836:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06618 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.246845:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06618 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.246852:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760771.246857:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06618 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.246867:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06618 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.246885:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06618 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.246894:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06618 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.247109:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493808128:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47061us (47404us total) trans 4294973753 rc 0/0 00000100:00100000:2.0:1789760771.247120:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6464 00000100:00100000:2.0:1789760771.247947:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781492911120 time 781541065559 state 0x2 ref 0 00000100:00100000:3.0:1789760771.259392:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760771.259494:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493808256 00000100:00100000:2.0:1789760771.259509:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493808256 00000100:00100000:2.0:1789760771.259563:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781492911120 time 781541065559 state 0x0 ref 1 00000100:00100000:2.0:1789760771.259578:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781541325975 time 781541065559 state 0x1 ref 1 00000100:00100000:2.0:1789760771.259661:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781541325975 time 781600695867 state 0x0 ref 1 00000100:00100000:2.0:1789760771.259674:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493808256:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760771.259736:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26357760, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:2.0:1789760771.259745:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0661f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.259761:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=781s v=(0 0) 00010000:00010000:2.0:1789760771.259765:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.259771:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0661f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.259788:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0661f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.317930:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973754 is committed 00002000:00010000:3.0:1789760771.318610:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26357760 -> 26361856 00010000:00010000:3.0:1789760771.318625:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0661f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.318638:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0661f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.318647:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0661f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.318654:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760771.318659:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0661f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.318667:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0661f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.318686:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0661f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.318693:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0661f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.318837:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493808256:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59160us (59449us total) trans 4294973754 rc 0/0 00000100:00100000:3.0:1789760771.318849:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6465 00000100:00100000:3.0:1789760771.318860:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781541325975 time 781600695867 state 0x2 ref 0 00000100:00100000:0.0:1789760771.325497:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760771.326478:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493808384 00000100:00100000:2.0:1789760771.326490:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493808384 00000100:00100000:2.0:1789760771.326557:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781541325975 time 781600695867 state 0x0 ref 1 00000100:00100000:2.0:1789760771.326572:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781600956283 time 781600695867 state 0x1 ref 1 00000100:00100000:2.0:1789760771.326612:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781600956283 time 781667647412 state 0x0 ref 1 00000100:00100000:2.0:1789760771.326630:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493808384:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760771.326685:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26361856, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:2.0:1789760771.326694:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06626 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.326710:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=781s v=(0 0) 00010000:00010000:2.0:1789760771.326715:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.326722:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06626 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.326735:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06626 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.364026:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973755 is committed 00002000:00010000:2.0:1789760771.364650:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26361856 -> 26365952 00010000:00010000:2.0:1789760771.364669:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06626 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.364682:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06626 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.364692:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06626 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.364699:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760771.364704:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06626 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.364715:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06626 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.364746:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06626 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.364755:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06626 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.364848:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493808384:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38216us (39357us total) trans 4294973755 rc 0/0 00000100:00100000:2.0:1789760771.364859:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6466 00000100:00100000:2.0:1789760771.364869:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781600956283 time 781667647412 state 0x2 ref 0 00000100:00100000:2.0:1789760771.372564:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760771.372683:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493808512 00000100:00100000:0.0:1789760771.372693:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493808512 00000100:00100000:0.0:1789760771.372892:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781600956283 time 781667647412 state 0x0 ref 1 00000100:00100000:0.0:1789760771.372908:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781667907828 time 781667647412 state 0x1 ref 1 00000100:00100000:0.0:1789760771.372930:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781667907828 time 781713966198 state 0x0 ref 1 00000100:00100000:0.0:1789760771.372945:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb14450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493808512:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760771.373041:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26365952, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:0.0:1789760771.373051:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e0662d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.373067:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=781s v=(0 0) 00010000:00010000:0.0:1789760771.373071:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.373078:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e0662d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.373092:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e0662d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.411250:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973756 is committed 00002000:00010000:1.0:1789760771.411405:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26365952 -> 26370048 00010000:00010000:1.0:1789760771.411421:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e0662d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.411434:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e0662d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.411444:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e0662d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.411452:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760771.411457:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e0662d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.411466:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e0662d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.411488:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e0662d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.411497:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e0662d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.411594:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb14450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493808512:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38649us (39035us total) trans 4294973756 rc 0/0 00000100:00100000:1.0:1789760771.411609:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6467 00000100:00100000:1.0:1789760771.411620:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781667907828 time 781713966198 state 0x2 ref 0 00000100:00100000:0.0:1789760771.416446:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760771.416565:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493808640 00000100:00100000:1.0:1789760771.416576:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493808640 00000100:00100000:1.0:1789760771.416628:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781667907828 time 781713966198 state 0x0 ref 1 00000100:00100000:1.0:1789760771.416642:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781714226614 time 781713966198 state 0x1 ref 1 00000100:00100000:1.0:1789760771.416675:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781714226614 time 781757710490 state 0x0 ref 1 00000100:00100000:1.0:1789760771.416687:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbeea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493808640:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760771.416736:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26370048, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:1.0:1789760771.416745:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06634 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.416761:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=781s v=(0 0) 00010000:00010000:1.0:1789760771.416765:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.416770:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06634 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.416783:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06634 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.464914:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973757 is committed 00002000:00010000:2.0:1789760771.465183:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26370048 -> 26374144 00010000:00010000:2.0:1789760771.465199:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06634 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.465212:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06634 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.465221:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06634 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.465233:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760771.465237:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06634 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.465246:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06634 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.465277:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06634 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.465285:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06634 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.465410:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbeea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493808640:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48721us (48970us total) trans 4294973757 rc 0/0 00000100:00100000:2.0:1789760771.465420:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6468 00000100:00100000:2.0:1789760771.465430:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781714226614 time 781757710490 state 0x2 ref 0 00000100:00100000:3.0:1789760771.469007:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760771.469135:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493808768 00000100:00100000:2.0:1789760771.469146:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493808768 00000100:00100000:2.0:1789760771.469199:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781714226614 time 781757710490 state 0x0 ref 1 00000100:00100000:2.0:1789760771.469214:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781757970906 time 781757710490 state 0x1 ref 1 00000100:00100000:2.0:1789760771.469244:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781757970906 time 781810279227 state 0x0 ref 1 00000100:00100000:2.0:1789760771.469268:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493808768:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760771.469317:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26374144, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:2.0:1789760771.469325:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0663b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.469341:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=781s v=(0 0) 00010000:00010000:2.0:1789760771.469345:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.469352:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0663b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.469366:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0663b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.498824:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973758 is committed 00002000:00010000:3.0:1789760771.498844:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26374144 -> 26378240 00010000:00010000:3.0:1789760771.498860:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0663b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.498873:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0663b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.498882:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0663b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.498890:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760771.498894:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0663b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.499035:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0663b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.499056:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0663b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.499064:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0663b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.499147:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493808768:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29878us (30182us total) trans 4294973758 rc 0/0 00000100:00100000:3.0:1789760771.499159:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6469 00000100:00100000:3.0:1789760771.499169:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781757970906 time 781810279227 state 0x2 ref 0 00000100:00100000:3.0:1789760771.505065:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760771.505393:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493808896 00000100:00100000:2.0:1789760771.505403:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493808896 00000100:00100000:2.0:1789760771.505464:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781757970906 time 781810279227 state 0x0 ref 1 00000100:00100000:2.0:1789760771.505480:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781810539643 time 781810279227 state 0x1 ref 1 00000100:00100000:2.0:1789760771.505508:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781810539643 time 781846544009 state 0x0 ref 1 00000100:00100000:2.0:1789760771.505521:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d8b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493808896:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760771.505578:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26378240, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:2.0:1789760771.505587:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06642 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.505608:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=781s v=(0 0) 00010000:00010000:2.0:1789760771.505613:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.505619:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06642 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.505632:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06642 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.546543:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973759 is committed 00002000:00010000:3.0:1789760771.546704:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26378240 -> 26382336 00010000:00010000:3.0:1789760771.546721:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06642 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.546732:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06642 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.546742:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06642 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.546749:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760771.546754:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06642 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.546763:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06642 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.546782:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06642 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.546791:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06642 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.546892:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d8b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493808896:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41368us (41925us total) trans 4294973759 rc 0/0 00000100:00100000:3.0:1789760771.546902:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6470 00000100:00100000:3.0:1789760771.546913:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781810539643 time 781846544009 state 0x2 ref 0 00000100:00100000:0.0:1789760771.553811:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760771.553952:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493809024 00000100:00100000:1.0:1789760771.554066:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493809024 00000100:00100000:1.0:1789760771.554124:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781810539643 time 781846544009 state 0x0 ref 1 00000100:00100000:1.0:1789760771.554138:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781846804425 time 781846544009 state 0x1 ref 1 00000100:00100000:1.0:1789760771.554460:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781846804425 time 781895494461 state 0x0 ref 1 00000100:00100000:1.0:1789760771.554476:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493809024:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760771.554524:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26382336, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:1.0:1789760771.554533:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06649 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.554561:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=781s v=(0 0) 00010000:00010000:1.0:1789760771.554567:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.554573:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06649 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.554587:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06649 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760771.588167:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26382336 -> 26386432 00010000:00010000:3.0:1789760771.588187:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06649 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.588200:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06649 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.588208:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06649 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.588215:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760771.588219:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06649 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.588227:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06649 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.588248:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06649 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.588270:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06649 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.588373:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493809024:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33895us (34566us total) trans 4294973760 rc 0/0 00000100:00100000:3.0:1789760771.588384:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6471 00000100:00100000:3.0:1789760771.588394:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781846804425 time 781895494461 state 0x2 ref 0 00000001:00080000:2.0:1789760771.588706:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973760 is committed 00000100:00100000:0.0:1789760771.591870:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760771.591922:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493809152 00000100:00100000:3.0:1789760771.591932:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493809152 00000100:00100000:3.0:1789760771.592140:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781846804425 time 781895494461 state 0x0 ref 1 00000100:00100000:3.0:1789760771.592157:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781895754877 time 781895494461 state 0x1 ref 1 00000100:00100000:2.0:1789760771.592241:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781895754877 time 781933274424 state 0x0 ref 1 00000100:00100000:2.0:1789760771.592269:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493809152:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760771.592329:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26386432, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xd007 00010000:00010000:2.0:1789760771.592338:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06650 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.592353:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=781s v=(0 0) 00010000:00010000:2.0:1789760771.592358:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.592364:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06650 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.592378:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06650 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.623612:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973761 is committed 00002000:00010000:2.0:1789760771.623706:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26386432 -> 26390528 00002000:00010000:2.0:1789760771.623710:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760770 -> 1789760771 00002000:00010000:2.0:1789760771.623712:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760770 -> 1789760771 00010000:00010000:2.0:1789760771.623724:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06650 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.623741:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06650 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.623748:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06650 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.623753:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760771.623757:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06650 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.623762:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06650 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.623778:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06650 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.623783:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06650 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.624233:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493809152:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31970us (32362us total) trans 4294973761 rc 0/0 00000100:00100000:2.0:1789760771.624246:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6472 00000100:00100000:2.0:1789760771.624268:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781895754877 time 781933274424 state 0x2 ref 0 00000100:00100000:0.0:1789760771.627629:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760771.627951:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493809280 00000100:00100000:3.0:1789760771.627990:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493809280 00000100:00100000:3.0:1789760771.628108:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781895754877 time 781933274424 state 0x0 ref 1 00000100:00100000:3.0:1789760771.628119:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781933534840 time 781933274424 state 0x1 ref 1 00000100:00100000:3.0:1789760771.628143:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781933534840 time 781969179268 state 0x0 ref 1 00000100:00100000:3.0:1789760771.628156:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493809280:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760771.628383:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26390528, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:3.0:1789760771.628396:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06657 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.628528:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760771.628534:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.628540:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06657 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.628559:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06657 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.642657:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760771.674511:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26390528 -> 26394624 00010000:00010000:3.0:1789760771.674530:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06657 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.674542:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06657 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.674553:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06657 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.674560:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760771.674565:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06657 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.674574:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06657 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.674601:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06657 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.674608:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06657 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.674718:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493809280:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46558us (47092us total) trans 4294973762 rc 0/0 00000100:00100000:3.0:1789760771.674730:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6473 00000100:00100000:3.0:1789760771.674748:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781933534840 time 781969179268 state 0x2 ref 0 00000001:00080000:2.0:1789760771.675697:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973762 is committed 00000100:00100000:0.0:1789760771.680026:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760771.680169:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493809408 00000100:00100000:2.0:1789760771.680180:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493809408 00000100:00100000:2.0:1789760771.680235:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781933534840 time 781969179268 state 0x0 ref 1 00000100:00100000:2.0:1789760771.680250:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781969439684 time 781969179268 state 0x1 ref 1 00000100:00100000:2.0:1789760771.680331:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781969439684 time 782021366390 state 0x0 ref 1 00000100:00100000:2.0:1789760771.680344:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493809408:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760771.680391:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26394624, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:2.0:1789760771.680400:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0665e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.680418:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=782s v=(0 0) 00010000:00010000:2.0:1789760771.680422:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.680427:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0665e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.680441:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0665e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760771.710525:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973763 is committed 00002000:00010000:2.0:1789760771.710641:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26394624 -> 26398720 00010000:00010000:2.0:1789760771.710657:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0665e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.710670:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0665e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.710680:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0665e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.710687:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760771.710692:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0665e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.710701:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0665e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.710721:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0665e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.710729:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0665e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.710835:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493809408:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30488us (30816us total) trans 4294973763 rc 0/0 00000100:00100000:2.0:1789760771.710845:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6474 00000100:00100000:2.0:1789760771.710855:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781969439684 time 782021366390 state 0x2 ref 0 00000100:00100000:1.0:1789760771.714516:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760771.714599:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493809536 00000100:00100000:2.0:1789760771.714610:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493809536 00000100:00100000:2.0:1789760771.714665:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 781969439684 time 782021366390 state 0x0 ref 1 00000100:00100000:2.0:1789760771.714681:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782021626806 time 782021366390 state 0x1 ref 1 00000100:00100000:2.0:1789760771.714711:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782021626806 time 782055746835 state 0x0 ref 1 00000100:00100000:2.0:1789760771.714723:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493809536:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760771.714771:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26398720, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:2.0:1789760771.714781:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06665 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.714797:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=782s v=(0 0) 00010000:00010000:2.0:1789760771.714802:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.715540:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06665 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.715559:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06665 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760771.776777:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26398720 -> 26402816 00010000:00010000:0.0:1789760771.776801:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06665 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.776816:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06665 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.776826:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06665 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.776846:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760771.776851:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06665 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.776861:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06665 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.776882:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06665 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.776890:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06665 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.777012:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493809536:12345-192.168.203.60@tcp:4:dd.0 Request processed in 62285us (62502us total) trans 4294973764 rc 0/0 00000100:00100000:0.0:1789760771.777023:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6475 00000100:00100000:0.0:1789760771.777032:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782021626806 time 782055746835 state 0x2 ref 0 00000001:00080000:1.0:1789760771.778576:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973764 is committed 00000100:00100000:3.0:1789760771.780012:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760771.780073:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493809664 00000100:00100000:0.0:1789760771.780084:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493809664 00000100:00100000:0.0:1789760771.780138:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782021626806 time 782055746835 state 0x0 ref 1 00000100:00100000:0.0:1789760771.780153:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782056007251 time 782055746835 state 0x1 ref 1 00000100:00100000:0.0:1789760771.780190:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782056007251 time 782121225536 state 0x0 ref 1 00000100:00100000:0.0:1789760771.780206:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493809664:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760771.780264:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26402816, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:0.0:1789760771.780272:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0666c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.780288:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=782s v=(0 0) 00010000:00010000:0.0:1789760771.780294:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.780300:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0666c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.780323:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0666c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.835293:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973765 is committed 00002000:00010000:1.0:1789760771.835644:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26402816 -> 26406912 00010000:00010000:1.0:1789760771.835661:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0666c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.835674:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0666c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.835684:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0666c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.835691:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760771.835695:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0666c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.835704:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0666c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.835723:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0666c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.835807:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0666c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.836327:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493809664:12345-192.168.203.60@tcp:4:dd.0 Request processed in 55694us (55895us total) trans 4294973765 rc 0/0 00000100:00100000:1.0:1789760771.836509:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6476 00000100:00100000:1.0:1789760771.836520:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782056007251 time 782121225536 state 0x2 ref 0 00000100:00100000:2.0:1789760771.840787:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760771.840859:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493809792 00000100:00100000:3.0:1789760771.840868:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493809792 00000100:00100000:3.0:1789760771.840922:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782056007251 time 782121225536 state 0x0 ref 1 00000100:00100000:3.0:1789760771.840937:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782121485952 time 782121225536 state 0x1 ref 1 00000100:00100000:3.0:1789760771.840995:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782121485952 time 782182030211 state 0x0 ref 1 00000100:00100000:3.0:1789760771.841008:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493809792:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760771.841057:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26406912, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:3.0:1789760771.841067:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06673 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.841089:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=782s v=(0 0) 00010000:00010000:3.0:1789760771.841093:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.841101:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06673 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.841114:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06673 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760771.877359:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973766 is committed 00002000:00010000:0.0:1789760771.877585:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26406912 -> 26411008 00010000:00010000:0.0:1789760771.877601:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06673 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.877615:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06673 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.877624:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06673 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.877632:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760771.877637:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06673 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.877647:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06673 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.877669:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06673 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.877677:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06673 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.877808:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493809792:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36797us (37076us total) trans 4294973766 rc 0/0 00000100:00100000:0.0:1789760771.877819:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6477 00000100:00100000:0.0:1789760771.877829:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782121485952 time 782182030211 state 0x2 ref 0 00000100:00100000:0.0:1789760771.882901:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760771.882999:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493809920 00000100:00100000:2.0:1789760771.883009:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493809920 00000100:00100000:2.0:1789760771.883069:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782121485952 time 782182030211 state 0x0 ref 1 00000100:00100000:2.0:1789760771.883085:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782182290627 time 782182030211 state 0x1 ref 1 00000100:00100000:2.0:1789760771.883123:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782182290627 time 782224158449 state 0x0 ref 1 00000100:00100000:2.0:1789760771.883137:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbedf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493809920:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760771.883187:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26411008, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:2.0:1789760771.883197:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0667a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.883213:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=782s v=(0 0) 00010000:00010000:2.0:1789760771.883218:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.883225:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0667a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.883238:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0667a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760771.918269:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26411008 -> 26415104 00010000:00010000:2.0:1789760771.918288:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0667a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.918301:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0667a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.918380:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0667a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.918388:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760771.918394:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0667a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.918403:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0667a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.918423:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0667a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.918431:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0667a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.918626:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973767 is committed 00000100:00100000:2.0:1789760771.918672:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbedf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493809920:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35387us (35629us total) trans 4294973767 rc 0/0 00000100:00100000:2.0:1789760771.918683:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6478 00000100:00100000:2.0:1789760771.918694:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782182290627 time 782224158449 state 0x2 ref 0 00000100:00100000:3.0:1789760771.922181:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760771.922292:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493810048 00000100:00100000:0.0:1789760771.922303:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493810048 00000100:00100000:0.0:1789760771.922367:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782182290627 time 782224158449 state 0x0 ref 1 00000100:00100000:0.0:1789760771.922382:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782224418865 time 782224158449 state 0x1 ref 1 00000100:00100000:0.0:1789760771.922450:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782224418865 time 782263485163 state 0x0 ref 1 00000100:00100000:0.0:1789760771.922462:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c89fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493810048:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760771.922516:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26415104, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:0.0:1789760771.922650:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06681 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.922667:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=782s v=(0 0) 00010000:00010000:0.0:1789760771.922672:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.922678:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06681 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.922692:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06681 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760771.955465:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26415104 -> 26419200 00010000:00010000:0.0:1789760771.955487:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06681 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.955501:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06681 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.955511:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06681 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.955519:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760771.955595:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06681 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.955606:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06681 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.955627:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06681 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.955634:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06681 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.955881:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c89fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493810048:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33415us (33704us total) trans 4294973768 rc 0/0 00000100:00100000:0.0:1789760771.955892:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6479 00000100:00100000:0.0:1789760771.955902:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782224418865 time 782263485163 state 0x2 ref 0 00000001:00080000:0.0:1789760771.955946:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973768 is committed 00000100:00100000:1.0:1789760771.965097:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760771.965200:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493810176 00000100:00100000:2.0:1789760771.965211:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493810176 00000100:00100000:2.0:1789760771.965414:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782224418865 time 782263485163 state 0x0 ref 1 00000100:00100000:2.0:1789760771.965430:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782263745579 time 782263485163 state 0x1 ref 1 00000100:00100000:2.0:1789760771.965501:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782263745579 time 782306536283 state 0x0 ref 1 00000100:00100000:2.0:1789760771.965516:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493810176:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760771.965590:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26419200, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:2.0:1789760771.965600:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06688 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.965615:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=782s v=(0 0) 00010000:00010000:2.0:1789760771.965620:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.965627:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06688 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.965640:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06688 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.016012:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973769 is committed 00002000:00010000:2.0:1789760772.016131:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26419200 -> 26423296 00010000:00010000:2.0:1789760772.016148:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06688 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.016161:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06688 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.016170:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06688 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.016177:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760772.016181:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06688 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.016189:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06688 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.016213:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06688 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.016220:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06688 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.016329:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493810176:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50811us (51236us total) trans 4294973769 rc 0/0 00000100:00100000:2.0:1789760772.016340:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6480 00000100:00100000:2.0:1789760772.016351:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782263745579 time 782306536283 state 0x2 ref 0 00000100:00100000:1.0:1789760772.022418:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760772.022499:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493810304 00000100:00100000:3.0:1789760772.022510:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493810304 00000100:00100000:3.0:1789760772.022631:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782263745579 time 782306536283 state 0x0 ref 1 00000100:00100000:3.0:1789760772.022646:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782306796699 time 782306536283 state 0x1 ref 1 00000100:00100000:3.0:1789760772.022722:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782306796699 time 782363756867 state 0x0 ref 1 00000100:00100000:3.0:1789760772.022734:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493810304:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760772.022781:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26423296, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:3.0:1789760772.022789:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0668f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.022806:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=782s v=(0 0) 00010000:00010000:3.0:1789760772.022811:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.022817:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0668f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.022836:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0668f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.048834:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973770 is committed 00002000:00010000:3.0:1789760772.049282:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26423296 -> 26427392 00010000:00010000:3.0:1789760772.049298:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0668f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.049311:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0668f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.049321:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0668f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.049328:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760772.049333:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0668f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.049358:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0668f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.049425:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0668f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.049435:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0668f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.049672:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493810304:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26935us (27260us total) trans 4294973770 rc 0/0 00000100:00100000:3.0:1789760772.049748:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6481 00000100:00100000:3.0:1789760772.049759:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782306796699 time 782363756867 state 0x2 ref 0 00000100:00100000:3.0:1789760772.056705:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760772.056819:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493810432 00000100:00100000:0.0:1789760772.056827:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493810432 00000100:00100000:0.0:1789760772.056879:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782306796699 time 782363756867 state 0x0 ref 1 00000100:00100000:0.0:1789760772.056895:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782364017283 time 782363756867 state 0x1 ref 1 00000100:00100000:0.0:1789760772.056928:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782364017283 time 782397963668 state 0x0 ref 1 00000100:00100000:0.0:1789760772.056941:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493810432:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760772.057033:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26427392, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:0.0:1789760772.057042:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06696 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.057060:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=782s v=(0 0) 00010000:00010000:0.0:1789760772.057063:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.057067:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06696 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.057078:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06696 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.090753:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973771 is committed 00002000:00010000:2.0:1789760772.090901:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26427392 -> 26431488 00010000:00010000:2.0:1789760772.090916:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06696 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.090929:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06696 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.090938:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06696 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.090945:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760772.090949:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06696 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.090981:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06696 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.091001:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06696 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.091010:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06696 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.091137:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493810432:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34193us (34437us total) trans 4294973771 rc 0/0 00000100:00100000:2.0:1789760772.091149:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6482 00000100:00100000:2.0:1789760772.091158:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782364017283 time 782397963668 state 0x2 ref 0 00000100:00100000:1.0:1789760772.095795:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760772.095912:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493810560 00000100:00100000:0.0:1789760772.095923:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493810560 00000100:00100000:0.0:1789760772.096047:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782364017283 time 782397963668 state 0x0 ref 1 00000100:00100000:0.0:1789760772.096062:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782398224084 time 782397963668 state 0x1 ref 1 00000100:00100000:0.0:1789760772.096093:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782398224084 time 782437128698 state 0x0 ref 1 00000100:00100000:0.0:1789760772.096105:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493810560:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760772.096151:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26431488, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:0.0:1789760772.096161:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0669d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.096176:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=782s v=(0 0) 00010000:00010000:0.0:1789760772.096180:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.096187:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0669d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.096205:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0669d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.124005:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973772 is committed 00002000:00010000:0.0:1789760772.124200:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26431488 -> 26435584 00010000:00010000:0.0:1789760772.124228:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0669d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.124242:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0669d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.124261:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0669d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.124268:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760772.124273:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0669d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.124282:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0669d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.124304:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0669d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.124312:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e0669d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.124415:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493810560:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28307us (28714us total) trans 4294973772 rc 0/0 00000100:00100000:0.0:1789760772.124426:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6483 00000100:00100000:0.0:1789760772.124436:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782398224084 time 782437128698 state 0x2 ref 0 00000100:00100000:1.0:1789760772.130660:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760772.130948:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493810688 00000100:00100000:2.0:1789760772.130993:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493810688 00000100:00100000:2.0:1789760772.131054:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782398224084 time 782437128698 state 0x0 ref 1 00000100:00100000:2.0:1789760772.131072:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782437389114 time 782437128698 state 0x1 ref 1 00000100:00100000:2.0:1789760772.131097:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782437389114 time 782472132609 state 0x0 ref 1 00000100:00100000:2.0:1789760772.131111:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493810688:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760772.131162:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26435584, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:2.0:1789760772.131171:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e066a4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.131187:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=782s v=(0 0) 00010000:00010000:2.0:1789760772.131192:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.131198:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e066a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.131213:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e066a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760772.182563:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26435584 -> 26439680 00010000:00010000:2.0:1789760772.182587:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e066a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.182602:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e066a4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.182611:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e066a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.182619:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760772.182623:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e066a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.182633:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e066a4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.182654:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e066a4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.182662:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e066a4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.182905:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493810688:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51790us (52248us total) trans 4294973773 rc 0/0 00000100:00100000:2.0:1789760772.182917:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6484 00000100:00100000:2.0:1789760772.182927:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782437389114 time 782472132609 state 0x2 ref 0 00000001:00080000:1.0:1789760772.183036:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973773 is committed 00000100:00100000:1.0:1789760772.191541:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760772.191708:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493810816 00000100:00100000:3.0:1789760772.191719:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493810816 00000100:00100000:3.0:1789760772.191773:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782437389114 time 782472132609 state 0x0 ref 1 00000100:00100000:3.0:1789760772.191796:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782472393025 time 782472132609 state 0x1 ref 1 00000100:00100000:3.0:1789760772.191827:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782472393025 time 782532862762 state 0x0 ref 1 00000100:00100000:3.0:1789760772.191840:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493810816:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760772.191891:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26439680, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:3.0:1789760772.191900:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e066ab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.191916:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=782s v=(0 0) 00010000:00010000:3.0:1789760772.191920:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.191927:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e066ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.191942:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e066ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760772.226466:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26439680 -> 26443776 00010000:00010000:2.0:1789760772.226486:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e066ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.226498:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e066ab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.226506:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e066ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.226514:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760772.226518:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e066ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.226526:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e066ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.226550:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e066ab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.226557:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e066ab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.226647:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493810816:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34805us (35113us total) trans 4294973774 rc 0/0 00000100:00100000:2.0:1789760772.226659:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6485 00000100:00100000:2.0:1789760772.226668:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782472393025 time 782532862762 state 0x2 ref 0 00000001:00080000:1.0:1789760772.227610:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973774 is committed 00000100:00100000:0.0:1789760772.229721:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760772.233448:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493810944 00000100:00100000:2.0:1789760772.233467:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493810944 00000100:00100000:2.0:1789760772.233533:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782472393025 time 782532862762 state 0x0 ref 1 00000100:00100000:2.0:1789760772.233550:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782533123178 time 782532862762 state 0x1 ref 1 00000100:00100000:2.0:1789760772.233589:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782533123178 time 782574623837 state 0x0 ref 1 00000100:00100000:2.0:1789760772.233603:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493810944:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760772.233652:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26443776, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:2.0:1789760772.233661:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e066b2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.233677:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=782s v=(0 0) 00010000:00010000:2.0:1789760772.233683:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.233690:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e066b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.233713:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e066b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.259171:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973775 is committed 00002000:00010000:0.0:1789760772.259365:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26443776 -> 26447872 00010000:00010000:0.0:1789760772.259380:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e066b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.259393:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e066b2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.259402:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e066b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.259410:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760772.259416:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e066b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.259426:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e066b2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.259445:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e066b2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.259458:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e066b2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.259552:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493810944:12345-192.168.203.60@tcp:4:dd.0 Request processed in 25946us (29835us total) trans 4294973775 rc 0/0 00000100:00100000:0.0:1789760772.259564:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6486 00000100:00100000:0.0:1789760772.259574:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782533123178 time 782574623837 state 0x2 ref 0 00000100:00100000:0.0:1789760772.265103:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760772.265210:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493811072 00000100:00100000:2.0:1789760772.265221:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493811072 00000100:00100000:2.0:1789760772.265287:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782533123178 time 782574623837 state 0x0 ref 1 00000100:00100000:2.0:1789760772.265302:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782574884253 time 782574623837 state 0x1 ref 1 00000100:00100000:2.0:1789760772.265328:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782574884253 time 782606363996 state 0x0 ref 1 00000100:00100000:2.0:1789760772.265339:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493811072:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760772.265384:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26447872, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:2.0:1789760772.265392:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066b9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.265406:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=782s v=(0 0) 00010000:00010000:2.0:1789760772.265410:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.265416:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.265429:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760772.325260:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26447872 -> 26451968 00010000:00010000:2.0:1789760772.325282:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.325296:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066b9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.325318:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.325326:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760772.325330:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.325339:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066b9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.325359:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066b9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.325367:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066b9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.325470:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493811072:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60126us (60372us total) trans 4294973776 rc 0/0 00000100:00100000:2.0:1789760772.325486:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6487 00000100:00100000:2.0:1789760772.325496:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782574884253 time 782606363996 state 0x2 ref 0 00000001:00080000:1.0:1789760772.326056:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973776 is committed 00000100:00100000:1.0:1789760772.330163:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760772.330258:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493811200 00000100:00100000:2.0:1789760772.330268:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493811200 00000100:00100000:2.0:1789760772.330326:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782574884253 time 782606363996 state 0x0 ref 1 00000100:00100000:2.0:1789760772.330341:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782606624412 time 782606363996 state 0x1 ref 1 00000100:00100000:2.0:1789760772.330367:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782606624412 time 782671402713 state 0x0 ref 1 00000100:00100000:2.0:1789760772.330380:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493811200:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760772.330429:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26451968, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:2.0:1789760772.330438:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e066c0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.330455:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=782s v=(0 0) 00010000:00010000:2.0:1789760772.330459:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.330465:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e066c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.330479:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e066c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.408518:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973777 is committed 00002000:00010000:2.0:1789760772.408623:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26451968 -> 26456064 00010000:00010000:2.0:1789760772.408637:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e066c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.408651:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e066c0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.408659:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e066c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.408667:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760772.408671:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e066c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.408680:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e066c0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.408699:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e066c0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.408706:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e066c0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.408909:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493811200:12345-192.168.203.60@tcp:4:dd.0 Request processed in 78526us (78750us total) trans 4294973777 rc 0/0 00000100:00100000:2.0:1789760772.408919:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6488 00000100:00100000:2.0:1789760772.408929:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782606624412 time 782671402713 state 0x2 ref 0 00000100:00100000:0.0:1789760772.412887:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760772.413147:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493811328 00000100:00100000:3.0:1789760772.413156:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493811328 00000100:00100000:3.0:1789760772.413206:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782606624412 time 782671402713 state 0x0 ref 1 00000100:00100000:3.0:1789760772.413221:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782671663129 time 782671402713 state 0x1 ref 1 00000100:00100000:3.0:1789760772.413256:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782671663129 time 782754283595 state 0x0 ref 1 00000100:00100000:3.0:1789760772.413269:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4badc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493811328:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760772.413402:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26456064, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:3.0:1789760772.413411:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e066c7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.413426:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=782s v=(0 0) 00010000:00010000:3.0:1789760772.413430:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.413436:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e066c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.413449:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e066c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760772.456670:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26456064 -> 26460160 00010000:00010000:3.0:1789760772.456689:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e066c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.456708:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e066c7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.456717:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e066c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.456735:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760772.456740:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e066c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.456750:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e066c7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.456771:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e066c7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.456779:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e066c7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.456868:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4badc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493811328:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43597us (43986us total) trans 4294973778 rc 0/0 00000100:00100000:3.0:1789760772.456879:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6489 00000100:00100000:3.0:1789760772.456888:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782671663129 time 782754283595 state 0x2 ref 0 00000001:00080000:0.0:1789760772.457087:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973778 is committed 00000100:00100000:1.0:1789760772.461349:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760772.461614:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493811456 00000100:00100000:0.0:1789760772.461625:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493811456 00000100:00100000:0.0:1789760772.461683:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782671663129 time 782754283595 state 0x0 ref 1 00000100:00100000:0.0:1789760772.461699:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782754544011 time 782754283595 state 0x1 ref 1 00000100:00100000:0.0:1789760772.461732:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782754544011 time 782802767542 state 0x0 ref 1 00000100:00100000:0.0:1789760772.461745:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493811456:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760772.461794:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26460160, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:0.0:1789760772.461803:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e066ce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.461819:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=782s v=(0 0) 00010000:00010000:0.0:1789760772.461824:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.461837:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e066ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.461853:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e066ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.490124:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973779 is committed 00002000:00010000:2.0:1789760772.490262:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26460160 -> 26464256 00010000:00010000:2.0:1789760772.490278:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e066ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.490290:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e066ce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.490297:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e066ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.490369:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760772.490375:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e066ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.490385:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e066ce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.490405:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e066ce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.490413:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e066ce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.490608:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493811456:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28860us (29264us total) trans 4294973779 rc 0/0 00000100:00100000:2.0:1789760772.490618:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6490 00000100:00100000:2.0:1789760772.490628:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782754544011 time 782802767542 state 0x2 ref 0 00000100:00100000:2.0:1789760772.498187:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760772.498300:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493811584 00000100:00100000:0.0:1789760772.498310:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493811584 00000100:00100000:0.0:1789760772.498365:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782754544011 time 782802767542 state 0x0 ref 1 00000100:00100000:0.0:1789760772.498381:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782803027958 time 782802767542 state 0x1 ref 1 00000100:00100000:0.0:1789760772.498406:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782803027958 time 782839442145 state 0x0 ref 1 00000100:00100000:0.0:1789760772.498419:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a37480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493811584:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760772.498467:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26464256, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:0.0:1789760772.498476:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e066d5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.498491:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=782s v=(0 0) 00010000:00010000:0.0:1789760772.498496:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.498502:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e066d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.498516:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e066d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760772.524177:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26464256 -> 26468352 00010000:00010000:1.0:1789760772.524201:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e066d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.524214:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e066d5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.524224:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e066d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.524232:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760772.524237:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e066d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.524289:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e066d5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.524316:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e066d5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.524323:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e066d5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.524412:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a37480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493811584:12345-192.168.203.60@tcp:4:dd.0 Request processed in 25991us (26231us total) trans 4294973780 rc 0/0 00000100:00100000:1.0:1789760772.524422:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6491 00000100:00100000:1.0:1789760772.524430:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782803027958 time 782839442145 state 0x2 ref 0 00000001:00080000:2.0:1789760772.524466:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973780 is committed 00000100:00100000:0.0:1789760772.527125:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760772.527268:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493811712 00000100:00100000:2.0:1789760772.527278:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493811712 00000100:00100000:2.0:1789760772.527338:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782803027958 time 782839442145 state 0x0 ref 1 00000100:00100000:2.0:1789760772.527351:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782839702561 time 782839442145 state 0x1 ref 1 00000100:00100000:2.0:1789760772.527385:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782839702561 time 782868420366 state 0x0 ref 1 00000100:00100000:2.0:1789760772.527397:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493811712:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760772.527449:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26468352, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:2.0:1789760772.527457:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066dc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.527473:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=782s v=(0 0) 00010000:00010000:2.0:1789760772.527477:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.527484:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.527499:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.561697:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973781 is committed 00002000:00010000:2.0:1789760772.561777:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26468352 -> 26472448 00010000:00010000:2.0:1789760772.561792:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.561810:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066dc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.561820:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.561827:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760772.561832:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.561841:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066dc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.561862:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066dc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.561869:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066dc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.562035:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493811712:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34635us (34913us total) trans 4294973781 rc 0/0 00000100:00100000:2.0:1789760772.562048:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6492 00000100:00100000:2.0:1789760772.562061:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782839702561 time 782868420366 state 0x2 ref 0 00000100:00100000:0.0:1789760772.566404:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760772.566643:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493811840 00000100:00100000:3.0:1789760772.566653:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493811840 00000100:00100000:3.0:1789760772.566708:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782839702561 time 782868420366 state 0x0 ref 1 00000100:00100000:3.0:1789760772.566723:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782868680782 time 782868420366 state 0x1 ref 1 00000100:00100000:3.0:1789760772.566756:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782868680782 time 782907791258 state 0x0 ref 1 00000100:00100000:3.0:1789760772.566769:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493811840:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760772.566814:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26472448, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:3.0:1789760772.566823:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e066e3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.566839:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=782s v=(0 0) 00010000:00010000:3.0:1789760772.566843:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.566849:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e066e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.566865:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e066e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.615624:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973782 is committed 00002000:00010000:0.0:1789760772.615707:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26472448 -> 26476544 00010000:00010000:0.0:1789760772.615724:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e066e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.615738:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e066e3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.615746:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e066e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.615754:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760772.615759:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e066e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.615767:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e066e3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.615787:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e066e3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.615796:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e066e3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.616134:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493811840:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49362us (49734us total) trans 4294973782 rc 0/0 00000100:00100000:0.0:1789760772.616146:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6493 00000100:00100000:0.0:1789760772.616155:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782868680782 time 782907791258 state 0x2 ref 0 00000100:00100000:1.0:1789760772.621434:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760772.622059:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493811968 00000100:00100000:2.0:1789760772.622069:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493811968 00000100:00100000:2.0:1789760772.622131:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782868680782 time 782907791258 state 0x0 ref 1 00000100:00100000:2.0:1789760772.622146:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782908051674 time 782907791258 state 0x1 ref 1 00000100:00100000:2.0:1789760772.622177:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782908051674 time 782963212404 state 0x0 ref 1 00000100:00100000:2.0:1789760772.622189:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493811968:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760772.622237:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26476544, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xd007 00010000:00010000:2.0:1789760772.622256:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066ea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.622272:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=782s v=(0 0) 00010000:00010000:2.0:1789760772.622277:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.622283:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.622296:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760772.653803:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26476544 -> 26480640 00002000:00010000:2.0:1789760772.653812:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760771 -> 1789760772 00002000:00010000:2.0:1789760772.653815:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760771 -> 1789760772 00010000:00010000:2.0:1789760772.653829:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.653843:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066ea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.653853:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.653860:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760772.653865:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.653875:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066ea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.653894:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066ea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.653903:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e066ea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.654047:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973783 is committed 00000100:00100000:2.0:1789760772.654215:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493811968:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32022us (32787us total) trans 4294973783 rc 0/0 00000100:00100000:2.0:1789760772.654226:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6494 00000100:00100000:2.0:1789760772.654236:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782908051674 time 782963212404 state 0x2 ref 0 00000100:00100000:2.0:1789760772.659254:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760772.659380:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493812096 00000100:00100000:1.0:1789760772.659390:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493812096 00000100:00100000:1.0:1789760772.659445:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782908051674 time 782963212404 state 0x0 ref 1 00000100:00100000:1.0:1789760772.659462:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782963472820 time 782963212404 state 0x1 ref 1 00000100:00100000:1.0:1789760772.659495:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782963472820 time 783000530906 state 0x0 ref 1 00000100:00100000:1.0:1789760772.659508:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a34e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493812096:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760772.659624:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26480640, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd007 00010000:00010000:1.0:1789760772.659633:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e066f1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.659649:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=783s v=(0 0) 00010000:00010000:1.0:1789760772.659654:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.659660:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e066f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.659677:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e066f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.666231:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760772.699952:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26480640 -> 26484736 00010000:00010000:0.0:1789760772.700006:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e066f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.700021:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e066f1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.700040:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e066f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.700048:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760772.700052:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e066f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.700061:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e066f1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.700081:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e066f1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.700094:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e066f1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.700185:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a34e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493812096:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40674us (40944us total) trans 4294973784 rc 0/0 00000100:00100000:0.0:1789760772.700197:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6495 00000100:00100000:0.0:1789760772.700206:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782963472820 time 783000530906 state 0x2 ref 0 00000001:00080000:3.0:1789760772.701829:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973784 is committed 00000100:00100000:2.0:1789760772.708213:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760772.708616:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493812224 00000100:00100000:0.0:1789760772.708626:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493812224 00000100:00100000:0.0:1789760772.708689:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 782963472820 time 783000530906 state 0x0 ref 1 00000100:00100000:0.0:1789760772.708704:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783000791322 time 783000530906 state 0x1 ref 1 00000100:00100000:0.0:1789760772.708741:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783000791322 time 783049776093 state 0x0 ref 1 00000100:00100000:0.0:1789760772.708754:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493812224:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760772.708805:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26484736, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd007 00010000:00010000:0.0:1789760772.708814:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e066f8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.708830:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=783s v=(0 0) 00010000:00010000:0.0:1789760772.708834:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.708841:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e066f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.708859:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e066f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.765833:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973785 is committed 00002000:00010000:2.0:1789760772.765956:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26484736 -> 26488832 00010000:00010000:2.0:1789760772.765993:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e066f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.766005:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e066f8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.766015:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e066f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.766022:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760772.766026:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e066f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.766035:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e066f8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.766054:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e066f8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.766061:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e066f8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.766156:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493812224:12345-192.168.203.60@tcp:4:dd.0 Request processed in 57399us (57949us total) trans 4294973785 rc 0/0 00000100:00100000:2.0:1789760772.766167:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6496 00000100:00100000:2.0:1789760772.766177:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783000791322 time 783049776093 state 0x2 ref 0 00000100:00100000:2.0:1789760772.769841:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760772.770171:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493812352 00000100:00100000:3.0:1789760772.770180:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493812352 00000100:00100000:3.0:1789760772.770367:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783000791322 time 783049776093 state 0x0 ref 1 00000100:00100000:3.0:1789760772.770384:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783050036509 time 783049776093 state 0x1 ref 1 00000100:00100000:3.0:1789760772.770624:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783050036509 time 783111657876 state 0x0 ref 1 00000100:00100000:3.0:1789760772.770694:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493812352:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760772.770747:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26488832, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd007 00010000:00010000:3.0:1789760772.770755:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e066ff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.770770:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=783s v=(0 0) 00010000:00010000:3.0:1789760772.770774:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.770779:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e066ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.770795:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e066ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.815833:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973786 is committed 00002000:00010000:2.0:1789760772.816001:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26488832 -> 26492928 00010000:00010000:2.0:1789760772.816017:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e066ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.816029:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e066ff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.816039:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e066ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.816047:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760772.816052:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e066ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.816061:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e066ff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.816077:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e066ff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.816082:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e066ff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.816156:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493812352:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45462us (46321us total) trans 4294973786 rc 0/0 00000100:00100000:2.0:1789760772.816167:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6497 00000100:00100000:2.0:1789760772.816177:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783050036509 time 783111657876 state 0x2 ref 0 00000100:00100000:0.0:1789760772.820932:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760772.821226:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493812480 00000100:00100000:2.0:1789760772.821236:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493812480 00000100:00100000:2.0:1789760772.821303:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783050036509 time 783111657876 state 0x0 ref 1 00000100:00100000:2.0:1789760772.821319:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783111918292 time 783111657876 state 0x1 ref 1 00000100:00100000:2.0:1789760772.821434:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783111918292 time 783162468564 state 0x0 ref 1 00000100:00100000:2.0:1789760772.821448:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493812480:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760772.821503:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26492928, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd007 00010000:00010000:2.0:1789760772.821524:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06706 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.821540:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=783s v=(0 0) 00010000:00010000:2.0:1789760772.821545:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.821552:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06706 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.821690:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06706 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.874445:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973787 is committed 00002000:00010000:0.0:1789760772.874540:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26492928 -> 26497024 00010000:00010000:0.0:1789760772.874552:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06706 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.874561:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06706 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.874567:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06706 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.874571:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760772.874574:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06706 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.874580:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06706 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.874594:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06706 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.874607:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06706 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.874739:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493812480:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53290us (53812us total) trans 4294973787 rc 0/0 00000100:00100000:0.0:1789760772.874748:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6498 00000100:00100000:0.0:1789760772.874755:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783111918292 time 783162468564 state 0x2 ref 0 00000100:00100000:1.0:1789760772.877991:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760772.878092:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493812608 00000100:00100000:0.0:1789760772.878315:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493812608 00000100:00100000:0.0:1789760772.878383:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783111918292 time 783162468564 state 0x0 ref 1 00000100:00100000:0.0:1789760772.878400:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783162728980 time 783162468564 state 0x1 ref 1 00000100:00100000:0.0:1789760772.878434:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783162728980 time 783219469511 state 0x0 ref 1 00000100:00100000:0.0:1789760772.878446:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c87a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493812608:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760772.878500:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26497024, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd007 00010000:00010000:0.0:1789760772.878510:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e0670d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.878526:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=783s v=(0 0) 00010000:00010000:0.0:1789760772.878531:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.878537:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e0670d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.878557:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e0670d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.918112:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973788 is committed 00002000:00010000:2.0:1789760772.918255:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26497024 -> 26501120 00010000:00010000:2.0:1789760772.918269:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e0670d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.918281:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e0670d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.918289:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e0670d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.918301:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760772.918305:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e0670d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.918313:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e0670d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.918332:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e0670d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.918339:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e0670d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.918437:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c87a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493812608:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39987us (40477us total) trans 4294973788 rc 0/0 00000100:00100000:2.0:1789760772.918448:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6499 00000100:00100000:2.0:1789760772.918457:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783162728980 time 783219469511 state 0x2 ref 0 00000100:00100000:2.0:1789760772.921275:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760772.921484:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493812736 00000100:00100000:0.0:1789760772.921494:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493812736 00000100:00100000:0.0:1789760772.921547:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783162728980 time 783219469511 state 0x0 ref 1 00000100:00100000:0.0:1789760772.921562:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783219729927 time 783219469511 state 0x1 ref 1 00000100:00100000:0.0:1789760772.921623:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783219729927 time 783262657756 state 0x0 ref 1 00000100:00100000:0.0:1789760772.921636:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493812736:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760772.921682:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26501120, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd007 00010000:00010000:0.0:1789760772.921691:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06714 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.921706:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=783s v=(0 0) 00010000:00010000:0.0:1789760772.921710:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.921715:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06714 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.921729:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06714 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.956103:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973789 is committed 00002000:00010000:0.0:1789760772.956208:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26501120 -> 26505216 00010000:00010000:0.0:1789760772.956223:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06714 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.956236:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06714 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.956254:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06714 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.956262:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760772.956267:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06714 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.956276:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06714 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.956303:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06714 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.956314:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06714 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.956403:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493812736:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34764us (35132us total) trans 4294973789 rc 0/0 00000100:00100000:0.0:1789760772.956413:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6500 00000100:00100000:0.0:1789760772.956423:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783219729927 time 783262657756 state 0x2 ref 0 00000100:00100000:1.0:1789760772.960333:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760772.960456:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493812864 00000100:00100000:3.0:1789760772.960465:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493812864 00000100:00100000:3.0:1789760772.960519:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783219729927 time 783262657756 state 0x0 ref 1 00000100:00100000:3.0:1789760772.960535:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783262918172 time 783262657756 state 0x1 ref 1 00000100:00100000:3.0:1789760772.960566:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783262918172 time 783301601712 state 0x0 ref 1 00000100:00100000:3.0:1789760772.960579:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493812864:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760772.960626:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26505216, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd007 00010000:00010000:3.0:1789760772.960636:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0671b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.960650:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=783s v=(0 0) 00010000:00010000:3.0:1789760772.960654:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.960659:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0671b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.960672:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0671b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.989150:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973790 is committed 00002000:00010000:2.0:1789760772.989353:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26505216 -> 26509312 00010000:00010000:2.0:1789760772.989370:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0671b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.989383:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0671b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.989392:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0671b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.989399:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760772.989403:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0671b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.989412:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0671b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.989433:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0671b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.989441:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e0671b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.989571:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493812864:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28989us (29242us total) trans 4294973790 rc 0/0 00000100:00100000:2.0:1789760772.989581:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6501 00000100:00100000:2.0:1789760772.989591:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783262918172 time 783301601712 state 0x2 ref 0 00000100:00100000:1.0:1789760772.993291:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760772.993451:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493812992 00000100:00100000:2.0:1789760772.993461:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493812992 00000100:00100000:2.0:1789760772.993517:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783262918172 time 783301601712 state 0x0 ref 1 00000100:00100000:2.0:1789760772.993532:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783301862128 time 783301601712 state 0x1 ref 1 00000100:00100000:2.0:1789760772.993597:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783301862128 time 783334632709 state 0x0 ref 1 00000100:00100000:2.0:1789760772.993612:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493812992:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760772.993657:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26509312, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd007 00010000:00010000:2.0:1789760772.993665:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06722 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.993679:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=783s v=(0 0) 00010000:00010000:2.0:1789760772.993683:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.993689:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06722 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.993702:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06722 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760773.022269:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26509312 -> 26513408 00010000:00010000:0.0:1789760773.022290:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06722 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.022312:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06722 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.022321:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06722 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.022328:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760773.022332:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06722 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.022340:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06722 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.022360:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06722 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.022368:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06722 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.022459:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493812992:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28845us (29169us total) trans 4294973791 rc 0/0 00000100:00100000:0.0:1789760773.022469:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6502 00000100:00100000:0.0:1789760773.022479:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783301862128 time 783334632709 state 0x2 ref 0 00000001:00080000:0.0:1789760773.023127:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973791 is committed 00000100:00100000:1.0:1789760773.027105:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760773.027403:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493813120 00000100:00100000:0.0:1789760773.027413:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493813120 00000100:00100000:0.0:1789760773.027514:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783301862128 time 783334632709 state 0x0 ref 1 00000100:00100000:0.0:1789760773.027528:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783334893125 time 783334632709 state 0x1 ref 1 00000100:00100000:0.0:1789760773.027558:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783334893125 time 783368593896 state 0x0 ref 1 00000100:00100000:0.0:1789760773.027570:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d08e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493813120:12345-192.168.203.60@tcp:4:dd.0 00000100:00080000:1.0:1789760773.027660:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789760773.027666:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26513408, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd007 00010000:00010000:0.0:1789760773.027675:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06729 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.027691:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=783s v=(0 0) 00010000:00010000:0.0:1789760773.027696:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.027702:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06729 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.027715:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06729 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.065863:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973792 is committed 00002000:00010000:1.0:1789760773.065982:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26513408 -> 26517504 00010000:00010000:1.0:1789760773.065998:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06729 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.066012:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06729 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.066021:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06729 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.066028:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760773.066033:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06729 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.066051:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06729 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.066070:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06729 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.066078:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06729 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.066161:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d08e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493813120:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38587us (39060us total) trans 4294973792 rc 0/0 00000100:00100000:1.0:1789760773.066170:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6503 00000100:00100000:1.0:1789760773.066180:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783334893125 time 783368593896 state 0x2 ref 0 00000100:00100000:1.0:1789760773.069869:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760773.070195:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493813248 00000100:00100000:0.0:1789760773.070205:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493813248 00000100:00100000:0.0:1789760773.070271:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783334893125 time 783368593896 state 0x0 ref 1 00000100:00100000:0.0:1789760773.070289:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783368854312 time 783368593896 state 0x1 ref 1 00000100:00100000:0.0:1789760773.070330:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783368854312 time 783411365513 state 0x0 ref 1 00000100:00100000:0.0:1789760773.070342:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493813248:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760773.070389:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26517504, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd007 00010000:00010000:0.0:1789760773.070397:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06730 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.070413:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=783s v=(0 0) 00010000:00010000:0.0:1789760773.070417:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.070424:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06730 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.070438:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06730 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760773.097336:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26517504 -> 26521600 00010000:00010000:1.0:1789760773.097356:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06730 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.097370:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06730 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.097380:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06730 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.097393:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760773.097398:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06730 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.097406:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06730 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.097424:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06730 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.097431:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06730 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.097521:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493813248:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27176us (27657us total) trans 4294973793 rc 0/0 00000100:00100000:1.0:1789760773.097532:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6504 00000100:00100000:1.0:1789760773.097542:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783368854312 time 783411365513 state 0x2 ref 0 00000001:00080000:3.0:1789760773.097591:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973793 is committed 00000100:00100000:0.0:1789760773.100845:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760773.101211:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493813376 00000100:00100000:2.0:1789760773.101221:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493813376 00000100:00100000:2.0:1789760773.101285:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783368854312 time 783411365513 state 0x0 ref 1 00000100:00100000:2.0:1789760773.101300:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783411625929 time 783411365513 state 0x1 ref 1 00000100:00100000:2.0:1789760773.101320:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783411625929 time 783442356776 state 0x0 ref 1 00000100:00100000:2.0:1789760773.101336:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493813376:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760773.101382:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26521600, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd007 00010000:00010000:2.0:1789760773.101390:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06737 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.101405:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=783s v=(0 0) 00010000:00010000:2.0:1789760773.101409:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.101416:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06737 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.101429:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06737 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.128830:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973794 is committed 00002000:00010000:0.0:1789760773.128993:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26521600 -> 26525696 00010000:00010000:0.0:1789760773.129009:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06737 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.129021:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06737 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.129027:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06737 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.129033:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760773.129036:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06737 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.129042:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06737 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.129057:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06737 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.129063:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06737 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.129147:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493813376:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27808us (28306us total) trans 4294973794 rc 0/0 00000100:00100000:0.0:1789760773.129158:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6505 00000100:00100000:0.0:1789760773.129167:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783411625929 time 783442356776 state 0x2 ref 0 00000100:00100000:2.0:1789760773.134212:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760773.134293:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493813504 00000100:00100000:0.0:1789760773.134311:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493813504 00000100:00100000:0.0:1789760773.134366:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783411625929 time 783442356776 state 0x0 ref 1 00000100:00100000:0.0:1789760773.134382:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783442617192 time 783442356776 state 0x1 ref 1 00000100:00100000:0.0:1789760773.134419:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783442617192 time 783475454458 state 0x0 ref 1 00000100:00100000:0.0:1789760773.134435:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4361500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493813504:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760773.134490:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26525696, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd007 00010000:00010000:0.0:1789760773.134499:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0673e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.134515:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=783s v=(0 0) 00010000:00010000:0.0:1789760773.134519:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.134525:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0673e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.134537:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0673e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.186194:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973795 is committed 00002000:00010000:1.0:1789760773.186337:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26525696 -> 26529792 00010000:00010000:1.0:1789760773.186352:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0673e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.186365:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0673e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.186378:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0673e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.186386:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760773.186391:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0673e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.186401:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0673e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.186420:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0673e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.186428:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0673e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.186524:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4361500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493813504:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52086us (52317us total) trans 4294973795 rc 0/0 00000100:00100000:1.0:1789760773.186534:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6506 00000100:00100000:1.0:1789760773.186544:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783442617192 time 783475454458 state 0x2 ref 0 00000100:00100000:1.0:1789760773.190201:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760773.190309:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493813632 00000100:00100000:0.0:1789760773.190319:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493813632 00000100:00100000:0.0:1789760773.190373:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783442617192 time 783475454458 state 0x0 ref 1 00000100:00100000:0.0:1789760773.190389:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783475714874 time 783475454458 state 0x1 ref 1 00000100:00100000:0.0:1789760773.190422:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783475714874 time 783531457678 state 0x0 ref 1 00000100:00100000:0.0:1789760773.190434:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493813632:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760773.190482:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26529792, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd007 00010000:00010000:0.0:1789760773.190492:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06745 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.190514:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=783s v=(0 0) 00010000:00010000:0.0:1789760773.190518:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.190524:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06745 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.190538:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06745 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.216588:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973796 is committed 00002000:00010000:1.0:1789760773.216697:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26529792 -> 26533888 00010000:00010000:1.0:1789760773.216712:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06745 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.216730:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06745 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.216739:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06745 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.216745:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760773.216750:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06745 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.216759:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06745 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.216778:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06745 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.216786:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06745 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.216909:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493813632:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26472us (26714us total) trans 4294973796 rc 0/0 00000100:00100000:1.0:1789760773.216919:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6507 00000100:00100000:1.0:1789760773.216928:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783475714874 time 783531457678 state 0x2 ref 0 00000100:00100000:3.0:1789760773.221034:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760773.221147:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493813760 00000100:00100000:1.0:1789760773.221158:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493813760 00000100:00100000:1.0:1789760773.221223:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783475714874 time 783531457678 state 0x0 ref 1 00000100:00100000:1.0:1789760773.221248:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783531718094 time 783531457678 state 0x1 ref 1 00000100:00100000:1.0:1789760773.221276:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783531718094 time 783562311218 state 0x0 ref 1 00000100:00100000:1.0:1789760773.221288:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4529c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493813760:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760773.221342:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26533888, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd007 00010000:00010000:1.0:1789760773.221350:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0674c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.221364:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=783s v=(0 0) 00010000:00010000:1.0:1789760773.221369:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.221375:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0674c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.221388:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0674c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.224629:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760773.224676:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760773.224716:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493814016 00000100:00100000:0.0:1789760773.224727:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493814016 00000100:00100000:0.0:1789760773.224786:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 512 00000100:00100000:0.0:1789760773.224796:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2367:x1876699493814016:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:0.0:1789760773.224997:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2367:x1876699493814016:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 201us (376us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760773.225005:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 512 00000100:00100000:1.0:1789760773.225429:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760773.225499:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493813888 00000100:00100000:1.0:1789760773.225504:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493813888 00000100:00100000:1.0:1789760773.225537:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 411 00000100:00100000:1.0:1789760773.225544:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d48c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2367:x1876699493813888:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:1.0:1789760773.225579:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d48c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2367:x1876699493813888:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 35us (903us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760773.225585:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 411 00000100:00080000:1.0:1789760773.225609:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760773.225652:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493814144 00000100:00100000:0.0:1789760773.225657:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493814144 00000100:00100000:0.0:1789760773.225729:0:9460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 513 00000100:00100000:0.0:1789760773.225736:0:9460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d0a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2367:x1876699493814144:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:0.0:1789760773.225776:0:9460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d0a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2367:x1876699493814144:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 40us (348us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760773.225782:0:9460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 513 00000100:00080000:1.0:1789760773.226055:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:3.0:1789760773.264343:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973797 is committed 00002000:00010000:0.0:1789760773.264409:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26533888 -> 26537984 00010000:00010000:0.0:1789760773.264426:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0674c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.264497:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0674c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.264507:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0674c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.264514:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760773.264519:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0674c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.264528:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0674c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.264547:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0674c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.264555:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0674c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.264636:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4529c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493813760:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43346us (43606us total) trans 4294973797 rc 0/0 00000100:00100000:0.0:1789760773.264647:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6508 00000100:00100000:0.0:1789760773.264657:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783531718094 time 783562311218 state 0x2 ref 0 00000100:00100000:1.0:1789760773.273629:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760773.273730:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493814272 00000100:00100000:2.0:1789760773.273740:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493814272 00000100:00100000:2.0:1789760773.273795:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783531718094 time 783562311218 state 0x0 ref 1 00000100:00100000:2.0:1789760773.273874:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783562571634 time 783562311218 state 0x1 ref 1 00000100:00100000:2.0:1789760773.274062:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783562571634 time 783615096053 state 0x0 ref 1 00000100:00100000:2.0:1789760773.274076:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493814272:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760773.274124:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26537984, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd007 00010000:00010000:2.0:1789760773.274133:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06753 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.274148:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=783s v=(0 0) 00010000:00010000:2.0:1789760773.274153:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.274160:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06753 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.274174:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06753 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.328855:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973798 is committed 00002000:00010000:3.0:1789760773.328991:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26537984 -> 26542080 00010000:00010000:3.0:1789760773.329006:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06753 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.329019:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06753 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.329028:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06753 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.329041:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760773.329046:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06753 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.329057:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06753 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.329077:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06753 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.329085:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06753 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.329168:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493814272:12345-192.168.203.60@tcp:4:dd.0 Request processed in 55090us (55545us total) trans 4294973798 rc 0/0 00000100:00100000:3.0:1789760773.329179:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6509 00000100:00100000:3.0:1789760773.329188:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783562571634 time 783615096053 state 0x2 ref 0 00000100:00100000:3.0:1789760773.333288:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760773.333663:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493814400 00000100:00100000:2.0:1789760773.333673:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493814400 00000100:00100000:2.0:1789760773.333730:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783562571634 time 783615096053 state 0x0 ref 1 00000100:00100000:2.0:1789760773.333745:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783615356469 time 783615096053 state 0x1 ref 1 00000100:00100000:2.0:1789760773.333779:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783615356469 time 783674814701 state 0x0 ref 1 00000100:00100000:2.0:1789760773.333790:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493814400:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760773.333835:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26542080, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd007 00010000:00010000:2.0:1789760773.333844:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0675a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.333860:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=783s v=(0 0) 00010000:00010000:2.0:1789760773.333865:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.333871:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0675a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.333885:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0675a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760773.374995:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26542080 -> 26546176 00010000:00010000:2.0:1789760773.375016:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0675a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.375028:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0675a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.375038:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0675a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.375045:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760773.375051:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0675a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.375060:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0675a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.375078:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0675a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.375086:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0675a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.375175:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493814400:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41381us (41890us total) trans 4294973799 rc 0/0 00000100:00100000:2.0:1789760773.375186:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6510 00000100:00100000:2.0:1789760773.375202:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783615356469 time 783674814701 state 0x2 ref 0 00000001:00080000:2.0:1789760773.375251:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973799 is committed 00000100:00100000:2.0:1789760773.380548:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760773.381136:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493814528 00000100:00100000:1.0:1789760773.381147:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493814528 00000100:00100000:1.0:1789760773.381202:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783615356469 time 783674814701 state 0x0 ref 1 00000100:00100000:1.0:1789760773.381217:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783675075117 time 783674814701 state 0x1 ref 1 00000100:00100000:1.0:1789760773.381261:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783675075117 time 783722295960 state 0x0 ref 1 00000100:00100000:1.0:1789760773.381274:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4363480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493814528:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760773.381324:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26546176, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd007 00010000:00010000:1.0:1789760773.381332:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06761 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.381348:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=783s v=(0 0) 00010000:00010000:1.0:1789760773.381353:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.381360:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06761 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.381374:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06761 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.409504:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973800 is committed 00002000:00010000:3.0:1789760773.409605:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26546176 -> 26550272 00010000:00010000:3.0:1789760773.409624:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06761 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.409639:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06761 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.409654:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06761 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.409661:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760773.409667:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06761 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.409677:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06761 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.409697:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06761 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.409706:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06761 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.409801:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4363480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493814528:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28524us (29258us total) trans 4294973800 rc 0/0 00000100:00100000:3.0:1789760773.409814:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6511 00000100:00100000:3.0:1789760773.409824:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783675075117 time 783722295960 state 0x2 ref 0 00000100:00100000:2.0:1789760773.418140:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760773.419983:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493814656 00000100:00100000:0.0:1789760773.419996:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493814656 00000100:00100000:0.0:1789760773.420061:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783675075117 time 783722295960 state 0x0 ref 1 00000100:00100000:0.0:1789760773.420077:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783722556376 time 783722295960 state 0x1 ref 1 00000100:00100000:0.0:1789760773.420324:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783722556376 time 783761145284 state 0x0 ref 1 00000100:00100000:0.0:1789760773.420343:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493814656:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760773.420395:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26550272, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd007 00010000:00010000:0.0:1789760773.420404:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06768 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.420426:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=783s v=(0 0) 00010000:00010000:0.0:1789760773.420431:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.420437:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06768 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.420451:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06768 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.447172:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973801 is committed 00002000:00010000:0.0:1789760773.447300:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26550272 -> 26554368 00010000:00010000:0.0:1789760773.447323:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06768 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.447337:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06768 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.447353:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06768 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.447360:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760773.447365:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06768 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.447374:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06768 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.447394:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06768 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.447403:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06768 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.447500:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493814656:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27155us (29366us total) trans 4294973801 rc 0/0 00000100:00100000:0.0:1789760773.447511:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6512 00000100:00100000:0.0:1789760773.447521:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783722556376 time 783761145284 state 0x2 ref 0 00000100:00100000:1.0:1789760773.453948:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760773.454133:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493814784 00000100:00100000:3.0:1789760773.454143:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493814784 00000100:00100000:2.0:1789760773.454423:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783722556376 time 783761145284 state 0x0 ref 1 00000100:00100000:2.0:1789760773.454439:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783761405700 time 783761145284 state 0x1 ref 1 00000100:00100000:2.0:1789760773.454512:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783761405700 time 783795547363 state 0x0 ref 1 00000100:00100000:2.0:1789760773.454525:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493814784:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760773.454574:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26554368, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd007 00010000:00010000:2.0:1789760773.454584:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0676f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.454600:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=783s v=(0 0) 00010000:00010000:2.0:1789760773.454605:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.454610:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0676f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.454624:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0676f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.502821:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973802 is committed 00002000:00010000:1.0:1789760773.502933:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26554368 -> 26558464 00010000:00010000:1.0:1789760773.502950:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0676f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.502986:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0676f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.502997:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0676f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.503005:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760773.503010:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0676f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.503020:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0676f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.503040:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0676f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.503047:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0676f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.503147:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493814784:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48619us (49203us total) trans 4294973802 rc 0/0 00000100:00100000:1.0:1789760773.503158:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6513 00000100:00100000:1.0:1789760773.503169:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783761405700 time 783795547363 state 0x2 ref 0 00000100:00100000:2.0:1789760773.509867:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760773.510046:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493814912 00000100:00100000:0.0:1789760773.510057:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493814912 00000100:00100000:0.0:1789760773.510165:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783761405700 time 783795547363 state 0x0 ref 1 00000100:00100000:0.0:1789760773.510348:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783795807779 time 783795547363 state 0x1 ref 1 00000100:00100000:0.0:1789760773.510382:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783795807779 time 783851417063 state 0x0 ref 1 00000100:00100000:0.0:1789760773.510395:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493814912:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760773.510444:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26558464, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd007 00010000:00010000:0.0:1789760773.510453:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06776 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.510469:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=783s v=(0 0) 00010000:00010000:0.0:1789760773.510473:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.510480:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06776 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.510494:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06776 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.543903:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973803 is committed 00002000:00010000:3.0:1789760773.544059:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26558464 -> 26562560 00010000:00010000:3.0:1789760773.544075:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06776 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.544087:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06776 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.544096:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06776 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.544103:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760773.544107:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06776 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.544117:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06776 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.544143:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06776 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.544151:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06776 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.544260:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493814912:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33862us (34396us total) trans 4294973803 rc 0/0 00000100:00100000:3.0:1789760773.544270:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6514 00000100:00100000:3.0:1789760773.544279:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783795807779 time 783851417063 state 0x2 ref 0 00000100:00100000:2.0:1789760773.549119:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760773.549215:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493815040 00000100:00100000:3.0:1789760773.549227:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493815040 00000100:00100000:3.0:1789760773.549294:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783795807779 time 783851417063 state 0x0 ref 1 00000100:00100000:3.0:1789760773.549310:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783851677479 time 783851417063 state 0x1 ref 1 00000100:00100000:3.0:1789760773.549493:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783851677479 time 783890526742 state 0x0 ref 1 00000100:00100000:3.0:1789760773.549507:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493815040:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760773.549558:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26562560, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd007 00010000:00010000:3.0:1789760773.549567:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0677d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.549582:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=783s v=(0 0) 00010000:00010000:3.0:1789760773.549587:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.549593:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0677d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.549607:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0677d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.585033:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973804 is committed 00002000:00010000:3.0:1789760773.585128:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26562560 -> 26566656 00010000:00010000:3.0:1789760773.585143:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0677d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.585157:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0677d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.585172:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0677d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.585179:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760773.585184:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0677d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.585193:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0677d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.585210:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0677d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.585218:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0677d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.585421:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493815040:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35912us (36307us total) trans 4294973804 rc 0/0 00000100:00100000:3.0:1789760773.585431:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6515 00000100:00100000:3.0:1789760773.585440:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783851677479 time 783890526742 state 0x2 ref 0 00000100:00100000:1.0:1789760773.592453:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760773.592996:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493815168 00000100:00100000:0.0:1789760773.593007:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493815168 00000100:00100000:0.0:1789760773.593065:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783851677479 time 783890526742 state 0x0 ref 1 00000100:00100000:0.0:1789760773.593080:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783890787158 time 783890526742 state 0x1 ref 1 00000100:00100000:0.0:1789760773.593110:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783890787158 time 783934145369 state 0x0 ref 1 00000100:00100000:0.0:1789760773.593131:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493815168:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760773.593185:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26566656, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xd007 00010000:00010000:0.0:1789760773.593194:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06784 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.593209:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=783s v=(0 0) 00010000:00010000:0.0:1789760773.593214:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.593220:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06784 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.593234:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06784 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.647519:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973805 is committed 00002000:00010000:0.0:1789760773.647587:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26566656 -> 26570752 00002000:00010000:0.0:1789760773.647592:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760772 -> 1789760773 00002000:00010000:0.0:1789760773.647595:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760772 -> 1789760773 00010000:00010000:0.0:1789760773.647608:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06784 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.647621:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06784 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.647629:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06784 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.647637:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760773.647643:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06784 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.647653:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06784 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.647672:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06784 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.647679:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06784 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.647774:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493815168:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54641us (55324us total) trans 4294973805 rc 0/0 00000100:00100000:0.0:1789760773.647784:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6516 00000100:00100000:0.0:1789760773.647795:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783890787158 time 783934145369 state 0x2 ref 0 00000100:00100000:0.0:1789760773.651278:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760773.651362:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493815296 00000100:00100000:1.0:1789760773.651372:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493815296 00000100:00100000:1.0:1789760773.651428:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783890787158 time 783934145369 state 0x0 ref 1 00000100:00100000:1.0:1789760773.651444:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783934405785 time 783934145369 state 0x1 ref 1 00000100:00100000:1.0:1789760773.651478:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783934405785 time 783992508975 state 0x0 ref 1 00000100:00100000:1.0:1789760773.651492:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493815296:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760773.651541:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26570752, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd007 00010000:00010000:1.0:1789760773.651550:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0678b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.651567:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=783s v=(0 0) 00010000:00010000:1.0:1789760773.651572:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.651578:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0678b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.651593:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0678b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.682351:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973806 is committed 00002000:00010000:3.0:1789760773.682502:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26570752 -> 26574848 00010000:00010000:3.0:1789760773.682517:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0678b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.682537:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0678b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.682547:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0678b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.682555:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760773.682560:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0678b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.682569:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0678b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.682591:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0678b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.682599:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0678b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.682693:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493815296:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31199us (31421us total) trans 4294973806 rc 0/0 00000100:00100000:3.0:1789760773.682703:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6517 00000100:00100000:3.0:1789760773.682713:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783934405785 time 783992508975 state 0x2 ref 0 00000100:00100000:3.0:1789760773.687838:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760773.688294:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493815424 00000100:00100000:0.0:1789760773.688304:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493815424 00000100:00100000:0.0:1789760773.688355:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783934405785 time 783992508975 state 0x0 ref 1 00000100:00100000:0.0:1789760773.688370:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783992769391 time 783992508975 state 0x1 ref 1 00000100:00100000:0.0:1789760773.688435:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783992769391 time 784029469925 state 0x0 ref 1 00000100:00100000:0.0:1789760773.688450:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493815424:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760773.688501:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26574848, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd007 00010000:00010000:0.0:1789760773.688510:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06792 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.688525:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=784s v=(0 0) 00010000:00010000:0.0:1789760773.688528:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.688535:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06792 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.688547:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06792 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.694317:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760773.739837:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973807 is committed 00002000:00010000:1.0:1789760773.740530:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26574848 -> 26578944 00010000:00010000:1.0:1789760773.740545:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06792 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.740558:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06792 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.740568:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06792 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.740576:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760773.740580:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06792 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.740589:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06792 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.740607:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06792 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.740614:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06792 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.740821:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493815424:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52368us (52986us total) trans 4294973807 rc 0/0 00000100:00100000:1.0:1789760773.740833:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6518 00000100:00100000:1.0:1789760773.740843:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783992769391 time 784029469925 state 0x2 ref 0 00000100:00100000:1.0:1789760773.750644:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760773.750851:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493815552 00000100:00100000:3.0:1789760773.750861:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493815552 00000100:00100000:3.0:1789760773.750933:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 783992769391 time 784029469925 state 0x0 ref 1 00000100:00100000:3.0:1789760773.750948:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784029730341 time 784029469925 state 0x1 ref 1 00000100:00100000:3.0:1789760773.751014:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784029730341 time 784092049046 state 0x0 ref 1 00000100:00100000:3.0:1789760773.751028:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493815552:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760773.751076:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26578944, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd007 00010000:00010000:3.0:1789760773.751084:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06799 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.751099:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=784s v=(0 0) 00010000:00010000:3.0:1789760773.751104:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.751110:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06799 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.751123:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06799 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.785151:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973808 is committed 00002000:00010000:1.0:1789760773.785248:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26578944 -> 26583040 00010000:00010000:1.0:1789760773.785263:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06799 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.785275:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06799 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.785284:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06799 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.785413:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760773.785419:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06799 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.785430:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06799 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.785452:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06799 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.785459:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06799 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.785674:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493815552:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34644us (35034us total) trans 4294973808 rc 0/0 00000100:00100000:1.0:1789760773.785685:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6519 00000100:00100000:1.0:1789760773.785695:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784029730341 time 784092049046 state 0x2 ref 0 00000100:00100000:0.0:1789760773.791713:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760773.791851:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493815680 00000100:00100000:1.0:1789760773.791861:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493815680 00000100:00100000:1.0:1789760773.791918:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784029730341 time 784092049046 state 0x0 ref 1 00000100:00100000:1.0:1789760773.791934:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784092309462 time 784092049046 state 0x1 ref 1 00000100:00100000:1.0:1789760773.792029:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784092309462 time 784133063793 state 0x0 ref 1 00000100:00100000:1.0:1789760773.792044:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493815680:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760773.792093:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26583040, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd007 00010000:00010000:1.0:1789760773.792102:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067a0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.792119:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=784s v=(0 0) 00010000:00010000:1.0:1789760773.792134:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.792141:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.792154:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.828042:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973809 is committed 00002000:00010000:1.0:1789760773.828227:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26583040 -> 26587136 00010000:00010000:1.0:1789760773.828253:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.828267:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067a0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.828276:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.828283:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760773.828288:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.828298:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067a0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.828319:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067a0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.828327:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067a0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.828433:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493815680:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36389us (36725us total) trans 4294973809 rc 0/0 00000100:00100000:1.0:1789760773.828447:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6520 00000100:00100000:1.0:1789760773.828457:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784092309462 time 784133063793 state 0x2 ref 0 00000100:00100000:3.0:1789760773.834743:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760773.834841:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493815808 00000100:00100000:0.0:1789760773.834851:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493815808 00000100:00100000:0.0:1789760773.834905:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784092309462 time 784133063793 state 0x0 ref 1 00000100:00100000:0.0:1789760773.834920:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784133324209 time 784133063793 state 0x1 ref 1 00000100:00100000:0.0:1789760773.835019:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784133324209 time 784176053819 state 0x0 ref 1 00000100:00100000:0.0:1789760773.835039:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493815808:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760773.835179:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26587136, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd007 00010000:00010000:0.0:1789760773.835189:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e067a7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.835210:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=784s v=(0 0) 00010000:00010000:0.0:1789760773.835214:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.835221:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e067a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.835234:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e067a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.877248:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973810 is committed 00002000:00010000:1.0:1789760773.877317:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26587136 -> 26591232 00010000:00010000:1.0:1789760773.877333:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e067a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.877346:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e067a7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.877355:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e067a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.877363:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760773.877368:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e067a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.877378:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e067a7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.877398:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e067a7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.877406:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e067a7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.877518:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493815808:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42476us (42779us total) trans 4294973810 rc 0/0 00000100:00100000:1.0:1789760773.877529:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6521 00000100:00100000:1.0:1789760773.877538:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784133324209 time 784176053819 state 0x2 ref 0 00000100:00100000:2.0:1789760773.881387:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760773.881508:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493815936 00000100:00100000:0.0:1789760773.881519:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493815936 00000100:00100000:0.0:1789760773.881573:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784133324209 time 784176053819 state 0x0 ref 1 00000100:00100000:0.0:1789760773.881596:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784176314235 time 784176053819 state 0x1 ref 1 00000100:00100000:0.0:1789760773.881625:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784176314235 time 784222660838 state 0x0 ref 1 00000100:00100000:0.0:1789760773.881639:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7aa7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493815936:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760773.881692:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26591232, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd007 00010000:00010000:0.0:1789760773.881701:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e067ae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.881717:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=784s v=(0 0) 00010000:00010000:0.0:1789760773.881721:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.881728:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e067ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.881747:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e067ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.935794:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973811 is committed 00002000:00010000:1.0:1789760773.935955:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26591232 -> 26595328 00010000:00010000:1.0:1789760773.935986:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e067ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.935998:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e067ae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.936008:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e067ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.936016:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760773.936020:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e067ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.936028:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e067ae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.936048:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e067ae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.936060:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e067ae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.936164:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7aa7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493815936:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54523us (54783us total) trans 4294973811 rc 0/0 00000100:00100000:1.0:1789760773.936174:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6522 00000100:00100000:1.0:1789760773.936184:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784176314235 time 784222660838 state 0x2 ref 0 00000100:00100000:2.0:1789760773.941075:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760773.941190:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493816064 00000100:00100000:1.0:1789760773.941200:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493816064 00000100:00100000:1.0:1789760773.941262:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784176314235 time 784222660838 state 0x0 ref 1 00000100:00100000:1.0:1789760773.941277:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784222921254 time 784222660838 state 0x1 ref 1 00000100:00100000:1.0:1789760773.941309:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784222921254 time 784282343904 state 0x0 ref 1 00000100:00100000:1.0:1789760773.941321:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493816064:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760773.941371:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26595328, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd007 00010000:00010000:1.0:1789760773.941381:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067b5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.941399:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=784s v=(0 0) 00010000:00010000:1.0:1789760773.941404:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.941410:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.941432:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760773.967547:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26595328 -> 26599424 00010000:00010000:0.0:1789760773.967566:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.967579:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067b5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.967587:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.967594:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760773.967599:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.967608:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.967627:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067b5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.967636:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067b5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.967652:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973812 is committed 00000100:00100000:0.0:1789760773.967768:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493816064:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26444us (26699us total) trans 4294973812 rc 0/0 00000100:00100000:0.0:1789760773.967779:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6523 00000100:00100000:0.0:1789760773.967789:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784222921254 time 784282343904 state 0x2 ref 0 00000100:00100000:2.0:1789760773.971526:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760773.971901:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493816192 00000100:00100000:1.0:1789760773.971911:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493816192 00000100:00100000:1.0:1789760773.973097:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784222921254 time 784282343904 state 0x0 ref 1 00000100:00100000:1.0:1789760773.973119:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784282604320 time 784282343904 state 0x1 ref 1 00000100:00100000:1.0:1789760773.973313:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784282604320 time 784314347641 state 0x0 ref 1 00000100:00100000:1.0:1789760773.973394:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb16850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493816192:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760773.973446:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26599424, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd007 00010000:00010000:1.0:1789760773.973455:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067bc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.973470:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=784s v=(0 0) 00010000:00010000:1.0:1789760773.973475:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.973481:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.973495:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.010998:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc2884380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519258752:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760774.011069:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760774.011185:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519258752 00000100:00100000:0.0:1789760774.011193:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519258752 00000100:00100000:0.0:1789760774.011267:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 212 00000100:00100000:0.0:1789760774.011668:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519258752:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760774.011690:0:6690:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528416768 pending 1703936 free 3831496704 avail 3798990848 00000001:00080000:3.0:1789760774.013705:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973813 is committed 00000100:00080000:3.0:1789760774.014601:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789760774.014746:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26599424 -> 26603520 00010000:00010000:1.0:1789760774.014762:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.014773:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067bc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.014781:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.014788:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760774.014793:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.014802:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067bc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.014822:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067bc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.014831:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067bc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.014985:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb16850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493816192:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41632us (43438us total) trans 4294973813 rc 0/0 00000100:00100000:1.0:1789760774.015000:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6524 00000100:00100000:1.0:1789760774.015009:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784282604320 time 784314347641 state 0x2 ref 0 00000100:00100000:0.0:1789760774.015630:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519258752:12345-0@lo:13:osp-pre-0-0.0 Request processed in 3963us (4559us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760774.015639:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 212 00000100:00100000:2.0:1789760774.017861:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc2884380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519258752:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760774.027445:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760774.027532:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493816320 00000100:00100000:0.0:1789760774.027542:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493816320 00000100:00100000:0.0:1789760774.027594:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784282604320 time 784314347641 state 0x0 ref 1 00000100:00100000:0.0:1789760774.027609:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784314608057 time 784314347641 state 0x1 ref 1 00000100:00100000:0.0:1789760774.027644:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784314608057 time 784368678704 state 0x0 ref 1 00000100:00100000:0.0:1789760774.027656:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493816320:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760774.027703:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26603520, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd007 00010000:00010000:0.0:1789760774.027712:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067c3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.027729:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=784s v=(0 0) 00010000:00010000:0.0:1789760774.027733:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.027740:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.027753:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.083090:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973814 is committed 00002000:00010000:1.0:1789760774.083449:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26603520 -> 26607616 00010000:00010000:1.0:1789760774.083464:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.083477:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067c3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.083486:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.083493:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760774.083498:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.083517:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067c3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.083538:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067c3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.083716:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067c3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.083809:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493816320:12345-192.168.203.60@tcp:4:dd.0 Request processed in 56150us (56370us total) trans 4294973814 rc 0/0 00000100:00100000:1.0:1789760774.083820:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6525 00000100:00100000:1.0:1789760774.083829:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784314608057 time 784368678704 state 0x2 ref 0 00000100:00100000:3.0:1789760774.088148:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760774.088276:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493816448 00000100:00100000:0.0:1789760774.088386:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493816448 00000100:00100000:0.0:1789760774.088441:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784314608057 time 784368678704 state 0x0 ref 1 00000100:00100000:0.0:1789760774.088456:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784368939120 time 784368678704 state 0x1 ref 1 00000100:00100000:0.0:1789760774.088487:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784368939120 time 784429522327 state 0x0 ref 1 00000100:00100000:0.0:1789760774.088499:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493816448:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760774.088552:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26607616, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd007 00010000:00010000:0.0:1789760774.088561:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e067ca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.088576:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=784s v=(0 0) 00010000:00010000:0.0:1789760774.088581:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.088586:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e067ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.088603:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e067ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.116625:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973815 is committed 00002000:00010000:3.0:1789760774.116760:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26607616 -> 26611712 00010000:00010000:3.0:1789760774.116780:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e067ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.116794:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e067ca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.116804:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e067ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.116812:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760774.116817:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e067ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.116826:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e067ca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.116846:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e067ca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.116854:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e067ca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.116947:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493816448:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28445us (28802us total) trans 4294973815 rc 0/0 00000100:00100000:3.0:1789760774.116957:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6526 00000100:00100000:3.0:1789760774.117078:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784368939120 time 784429522327 state 0x2 ref 0 00000100:00100000:2.0:1789760774.119641:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760774.120040:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493816576 00000100:00100000:1.0:1789760774.120050:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493816576 00000100:00100000:1.0:1789760774.120102:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784368939120 time 784429522327 state 0x0 ref 1 00000100:00100000:1.0:1789760774.120117:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784429782743 time 784429522327 state 0x1 ref 1 00000100:00100000:1.0:1789760774.120181:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784429782743 time 784461216390 state 0x0 ref 1 00000100:00100000:1.0:1789760774.120194:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8794700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493816576:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760774.120257:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26611712, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd007 00010000:00010000:1.0:1789760774.120266:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067d1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.120294:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=784s v=(0 0) 00010000:00010000:1.0:1789760774.120299:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.120305:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.120318:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760774.147032:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26611712 -> 26615808 00010000:00010000:0.0:1789760774.147054:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.147068:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067d1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.147077:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.147085:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760774.147089:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.147098:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067d1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.147117:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067d1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.147124:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067d1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.147231:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8794700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493816576:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27033us (27590us total) trans 4294973816 rc 0/0 00000100:00100000:0.0:1789760774.147253:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6527 00000100:00100000:0.0:1789760774.147263:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784429782743 time 784461216390 state 0x2 ref 0 00000001:00080000:1.0:1789760774.147776:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973816 is committed 00000100:00100000:2.0:1789760774.152097:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760774.152178:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493816704 00000100:00100000:0.0:1789760774.152188:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493816704 00000100:00100000:0.0:1789760774.152251:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784429782743 time 784461216390 state 0x0 ref 1 00000100:00100000:0.0:1789760774.152266:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784461476806 time 784461216390 state 0x1 ref 1 00000100:00100000:0.0:1789760774.152304:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784461476806 time 784493338963 state 0x0 ref 1 00000100:00100000:0.0:1789760774.152317:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8796a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493816704:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760774.152367:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26615808, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd007 00010000:00010000:0.0:1789760774.152376:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e067d8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.152391:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=784s v=(0 0) 00010000:00010000:0.0:1789760774.152400:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.152406:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e067d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.152420:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e067d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.187461:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973817 is committed 00002000:00010000:1.0:1789760774.187558:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26615808 -> 26619904 00010000:00010000:1.0:1789760774.187574:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e067d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.187587:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e067d8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.187596:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e067d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.187604:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760774.187608:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e067d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.187618:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e067d8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.187639:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e067d8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.187647:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e067d8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.187861:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8796a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493816704:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35540us (35766us total) trans 4294973817 rc 0/0 00000100:00100000:1.0:1789760774.187874:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6528 00000100:00100000:1.0:1789760774.187885:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784461476806 time 784493338963 state 0x2 ref 0 00000100:00100000:2.0:1789760774.190699:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760774.193179:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493816832 00000100:00100000:1.0:1789760774.193193:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493816832 00000100:00100000:1.0:1789760774.193268:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784461476806 time 784493338963 state 0x0 ref 1 00000100:00100000:1.0:1789760774.193285:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784493599379 time 784493338963 state 0x1 ref 1 00000100:00100000:0.0:1789760774.193388:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784493599379 time 784534421108 state 0x0 ref 1 00000100:00100000:0.0:1789760774.193406:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3663480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493816832:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760774.193458:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26619904, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd007 00010000:00010000:0.0:1789760774.193466:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067df lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.193482:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=784s v=(0 0) 00010000:00010000:0.0:1789760774.193486:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.193492:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.193507:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760774.221025:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26619904 -> 26624000 00000001:00080000:2.0:1789760774.221041:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973818 is committed 00010000:00010000:0.0:1789760774.221045:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.221058:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067df lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.221067:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.221081:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760774.221085:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.221094:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067df lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.221113:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067df lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.221121:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067df lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.221338:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3663480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493816832:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27808us (30517us total) trans 4294973818 rc 0/0 00000100:00100000:0.0:1789760774.221351:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6529 00000100:00100000:0.0:1789760774.221361:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784493599379 time 784534421108 state 0x2 ref 0 00000100:00100000:2.0:1789760774.224159:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760774.224269:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493816960 00000100:00100000:3.0:1789760774.224399:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493816960 00000100:00100000:3.0:1789760774.224487:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784493599379 time 784534421108 state 0x0 ref 1 00000100:00100000:3.0:1789760774.224508:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784534681524 time 784534421108 state 0x1 ref 1 00000100:00100000:3.0:1789760774.224543:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784534681524 time 784565578406 state 0x0 ref 1 00000100:00100000:3.0:1789760774.224561:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3662d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493816960:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760774.224631:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26624000, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd007 00010000:00010000:3.0:1789760774.224642:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e067e6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.224661:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=784s v=(0 0) 00010000:00010000:3.0:1789760774.224666:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.224674:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e067e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.224691:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e067e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.253813:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973819 is committed 00002000:00010000:1.0:1789760774.254082:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26624000 -> 26628096 00010000:00010000:1.0:1789760774.254210:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e067e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.254224:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e067e6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.254243:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e067e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.254251:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760774.254256:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e067e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.254266:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e067e6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.254411:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e067e6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.254419:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e067e6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.254510:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3662d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493816960:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29949us (30354us total) trans 4294973819 rc 0/0 00000100:00100000:1.0:1789760774.254521:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6530 00000100:00100000:1.0:1789760774.254530:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784534681524 time 784565578406 state 0x2 ref 0 00000100:00100000:2.0:1789760774.258637:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760774.259313:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493817088 00000100:00100000:1.0:1789760774.259326:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493817088 00000100:00100000:1.0:1789760774.259384:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784534681524 time 784565578406 state 0x0 ref 1 00000100:00100000:1.0:1789760774.259399:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784565838822 time 784565578406 state 0x1 ref 1 00000100:00100000:1.0:1789760774.259421:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784565838822 time 784600456453 state 0x0 ref 1 00000100:00100000:1.0:1789760774.259434:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3662680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493817088:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760774.259480:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26628096, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd007 00010000:00010000:1.0:1789760774.259489:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067ed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.259505:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=784s v=(0 0) 00010000:00010000:1.0:1789760774.259509:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.259515:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.259535:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.287686:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973820 is committed 00002000:00010000:2.0:1789760774.287775:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26628096 -> 26632192 00010000:00010000:2.0:1789760774.287795:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.287811:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067ed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.287820:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.287828:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760774.287833:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.287842:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067ed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.287866:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067ed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.287874:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e067ed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.288134:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3662680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493817088:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28696us (29505us total) trans 4294973820 rc 0/0 00000100:00100000:2.0:1789760774.288145:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6531 00000100:00100000:2.0:1789760774.288157:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784565838822 time 784600456453 state 0x2 ref 0 00000100:00100000:2.0:1789760774.294741:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760774.295511:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493817216 00000100:00100000:1.0:1789760774.295526:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493817216 00000100:00100000:1.0:1789760774.295601:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784565838822 time 784600456453 state 0x0 ref 1 00000100:00100000:1.0:1789760774.295619:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784600716869 time 784600456453 state 0x1 ref 1 00000100:00100000:1.0:1789760774.295657:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784600716869 time 784636692444 state 0x0 ref 1 00000100:00100000:1.0:1789760774.295670:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3662300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493817216:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760774.295734:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26632192, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd007 00010000:00010000:1.0:1789760774.295744:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067f4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.295764:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=784s v=(0 0) 00010000:00010000:1.0:1789760774.295769:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.295777:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.295792:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.331534:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973821 is committed 00002000:00010000:1.0:1789760774.331847:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26632192 -> 26636288 00010000:00010000:1.0:1789760774.331869:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.331886:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067f4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.331896:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.331906:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760774.331912:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.331923:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067f4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.331948:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067f4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.331956:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e067f4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.332108:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3662300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493817216:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36434us (37376us total) trans 4294973821 rc 0/0 00000100:00100000:1.0:1789760774.332122:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6532 00000100:00100000:1.0:1789760774.332135:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784600716869 time 784636692444 state 0x2 ref 0 00000100:00100000:3.0:1789760774.336091:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760774.336210:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493817344 00000100:00100000:0.0:1789760774.336222:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493817344 00000100:00100000:0.0:1789760774.336316:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784600716869 time 784636692444 state 0x0 ref 1 00000100:00100000:0.0:1789760774.336335:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784636952860 time 784636692444 state 0x1 ref 1 00000100:00100000:0.0:1789760774.336370:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784636952860 time 784677404866 state 0x0 ref 1 00000100:00100000:0.0:1789760774.336388:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4529c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493817344:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760774.336457:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26636288, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd007 00010000:00010000:0.0:1789760774.336572:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067fb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.336593:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=784s v=(0 0) 00010000:00010000:0.0:1789760774.336598:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.336606:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.336623:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.396149:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973822 is committed 00002000:00010000:0.0:1789760774.396497:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26636288 -> 26640384 00010000:00010000:0.0:1789760774.396528:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.396544:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067fb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.396553:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.396560:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760774.396566:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.396578:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.396601:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067fb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.396608:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e067fb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.396728:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4529c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493817344:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60335us (60646us total) trans 4294973822 rc 0/0 00000100:00100000:0.0:1789760774.396881:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6533 00000100:00100000:0.0:1789760774.396894:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784636952860 time 784677404866 state 0x2 ref 0 00000100:00100000:2.0:1789760774.402432:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760774.402540:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493817472 00000100:00100000:3.0:1789760774.402552:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493817472 00000100:00100000:3.0:1789760774.402627:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784636952860 time 784677404866 state 0x0 ref 1 00000100:00100000:3.0:1789760774.402645:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784677665282 time 784677404866 state 0x1 ref 1 00000100:00100000:3.0:1789760774.402751:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784677665282 time 784743785725 state 0x0 ref 1 00000100:00100000:3.0:1789760774.402767:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3661500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493817472:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760774.402834:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26640384, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd007 00010000:00010000:3.0:1789760774.402844:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06802 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.402863:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=784s v=(0 0) 00010000:00010000:3.0:1789760774.402867:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.402875:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06802 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.402890:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06802 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.464418:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973823 is committed 00002000:00010000:0.0:1789760774.464783:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26640384 -> 26644480 00010000:00010000:0.0:1789760774.464803:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06802 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.464819:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06802 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.464829:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06802 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.464837:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760774.464887:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06802 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.464898:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06802 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.464922:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06802 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.464930:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06802 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.465184:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3661500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493817472:12345-192.168.203.60@tcp:4:dd.0 Request processed in 62411us (62759us total) trans 4294973823 rc 0/0 00000100:00100000:0.0:1789760774.465196:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6534 00000100:00100000:0.0:1789760774.465208:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784677665282 time 784743785725 state 0x2 ref 0 00000100:00100000:3.0:1789760774.474462:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760774.474746:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493817600 00000100:00100000:2.0:1789760774.474759:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493817600 00000100:00100000:2.0:1789760774.474832:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784677665282 time 784743785725 state 0x0 ref 1 00000100:00100000:2.0:1789760774.474859:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784744046141 time 784743785725 state 0x1 ref 1 00000100:00100000:2.0:1789760774.474896:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784744046141 time 784815930878 state 0x0 ref 1 00000100:00100000:2.0:1789760774.474912:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493817600:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760774.475016:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26644480, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd007 00010000:00010000:2.0:1789760774.475026:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06809 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.475047:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=784s v=(0 0) 00010000:00010000:2.0:1789760774.475052:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.475060:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06809 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.475077:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06809 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760774.511177:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26644480 -> 26648576 00010000:00010000:3.0:1789760774.511205:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06809 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.511220:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06809 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.511239:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06809 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.511255:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760774.511261:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06809 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.511271:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06809 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.511304:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06809 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.511314:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06809 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.511442:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493817600:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36526us (36989us total) trans 4294973824 rc 0/0 00000100:00100000:3.0:1789760774.511454:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6535 00000100:00100000:3.0:1789760774.511467:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784744046141 time 784815930878 state 0x2 ref 0 00000001:00080000:2.0:1789760774.512190:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973824 is committed 00000100:00100000:3.0:1789760774.515023:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760774.515575:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493817728 00000100:00100000:1.0:1789760774.515589:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493817728 00000100:00100000:1.0:1789760774.515663:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784744046141 time 784815930878 state 0x0 ref 1 00000100:00100000:1.0:1789760774.515683:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784816191294 time 784815930878 state 0x1 ref 1 00000100:00100000:1.0:1789760774.515768:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784816191294 time 784856802360 state 0x0 ref 1 00000100:00100000:1.0:1789760774.515783:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493817728:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760774.515893:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26648576, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd007 00010000:00010000:1.0:1789760774.515905:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06810 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.515925:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=784s v=(0 0) 00010000:00010000:1.0:1789760774.515930:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.515937:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06810 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.515953:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06810 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.522836:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc2887b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3305:1876699519259264:0@lo:400:kworker.0 00000100:00100000:1.0:1789760774.522943:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760774.523043:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1789760774.523049:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc2884a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3305:1876699519258880:0@lo:400:kworker.0 00000100:00100000:1.0:1789760774.523083:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760774.523109:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc2886680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519259392:0@lo:400:kworker.0 00000100:00100000:1.0:1789760774.523139:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760774.523170:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc2886a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876699519259008:0@lo:400:kworker.0 00000100:00100000:1.0:1789760774.523199:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760774.523223:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760774.523229:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc2885c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3303:1876699519259136:0@lo:400:kworker.0 00000100:00100000:1.0:1789760774.523265:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760774.523319:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519259264 00000100:00100000:1.0:1789760774.523329:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519259264 00000100:00100000:1.0:1789760774.523396:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519259008 00000100:00100000:1.0:1789760774.523399:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519259008 00000100:00100000:1.0:1789760774.523421:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519259136 00000100:00100000:1.0:1789760774.523423:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519259136 00000100:00100000:1.0:1789760774.523447:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 514 00000100:00100000:1.0:1789760774.523456:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3305:x1876699519259264:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760774.523521:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3305:x1876699519259264:12345-0@lo:400:kworker.0 Request processed in 66us (578us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760774.523529:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 514 00000100:00100000:1.0:1789760774.523547:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 515 00000100:00100000:1.0:1789760774.523552:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876699519259008:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760774.523582:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876699519259008:12345-0@lo:400:kworker.0 Request processed in 30us (382us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760774.523588:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 515 00000100:00100000:1.0:1789760774.523600:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 516 00000100:00100000:1.0:1789760774.523605:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876699519259136:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760774.523634:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876699519259136:12345-0@lo:400:kworker.0 Request processed in 29us (368us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760774.523644:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 516 00000100:00100000:1.0:1789760774.523664:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519258880 00000100:00100000:1.0:1789760774.523667:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519258880 00000100:00100000:1.0:1789760774.523693:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 412 00000100:00100000:1.0:1789760774.523699:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d4a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3305:x1876699519258880:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760774.523727:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d4a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3305:x1876699519258880:12345-0@lo:400:kworker.0 Request processed in 29us (644us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760774.523734:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 412 00000100:00100000:1.0:1789760774.523765:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519259392 00000100:00100000:1.0:1789760774.523768:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519259392 00000100:00100000:1.0:1789760774.523804:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 184 00000100:00100000:1.0:1789760774.523811:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519259392:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760774.523846:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519259392:12345-0@lo:400:kworker.0 Request processed in 36us (706us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760774.523852:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 184 00000100:00080000:1.0:1789760774.523878:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789760774.523884:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760774.524131:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc2885c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3303:1876699519259136:0@lo:400:kworker.0 00000100:00100000:1.0:1789760774.524148:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc2886a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876699519259008:0@lo:400:kworker.0 00000100:00100000:1.0:1789760774.524187:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc2884a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3305:1876699519258880:0@lo:400:kworker.0 00000100:00100000:1.0:1789760774.524200:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc2886680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519259392:0@lo:400:kworker.0 00000100:00100000:2.0:1789760774.526518:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc2887b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3305:1876699519259264:0@lo:400:kworker.0 00002000:00010000:2.0:1789760774.548195:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26648576 -> 26652672 00010000:00010000:2.0:1789760774.548226:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06810 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.548255:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06810 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.548265:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06810 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.548274:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760774.548280:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06810 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.548298:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06810 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.548322:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06810 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.548330:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06810 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.548460:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493817728:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32671us (33504us total) trans 4294973825 rc 0/0 00000100:00100000:2.0:1789760774.548472:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6536 00000100:00100000:2.0:1789760774.548486:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784816191294 time 784856802360 state 0x2 ref 0 00000001:00080000:3.0:1789760774.550390:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973825 is committed 00000100:00100000:1.0:1789760774.552923:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760774.553087:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493817856 00000100:00100000:0.0:1789760774.553100:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493817856 00000100:00100000:0.0:1789760774.553175:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784816191294 time 784856802360 state 0x0 ref 1 00000100:00100000:0.0:1789760774.553195:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784857062776 time 784856802360 state 0x1 ref 1 00000100:00100000:0.0:1789760774.553291:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784857062776 time 784894317455 state 0x0 ref 1 00000100:00100000:0.0:1789760774.553306:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493817856:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760774.553371:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26652672, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd007 00010000:00010000:0.0:1789760774.553382:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06817 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.553402:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=784s v=(0 0) 00010000:00010000:0.0:1789760774.553407:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.553414:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06817 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.553430:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06817 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.592694:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973826 is committed 00002000:00010000:0.0:1789760774.592823:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26652672 -> 26656768 00010000:00010000:0.0:1789760774.592843:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06817 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.592859:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06817 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.592870:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06817 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.592878:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760774.592884:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06817 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.592895:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06817 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.592927:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06817 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.592936:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06817 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.593748:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493817856:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40434us (40828us total) trans 4294973826 rc 0/0 00000100:00100000:0.0:1789760774.593763:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6537 00000100:00100000:0.0:1789760774.593775:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784857062776 time 784894317455 state 0x2 ref 0 00000100:00100000:0.0:1789760774.596440:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760774.596694:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493817984 00000100:00100000:3.0:1789760774.596706:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493817984 00000100:00100000:3.0:1789760774.596829:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784857062776 time 784894317455 state 0x0 ref 1 00000100:00100000:3.0:1789760774.596847:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784894577871 time 784894317455 state 0x1 ref 1 00000100:00100000:3.0:1789760774.596920:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784894577871 time 784937954399 state 0x0 ref 1 00000100:00100000:3.0:1789760774.596934:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493817984:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760774.597025:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26656768, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xd007 00010000:00010000:3.0:1789760774.597035:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0681e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.597055:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=784s v=(0 0) 00010000:00010000:3.0:1789760774.597060:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.597068:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0681e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.597084:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0681e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.633814:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973827 is committed 00002000:00010000:0.0:1789760774.634015:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26656768 -> 26660864 00002000:00010000:0.0:1789760774.634021:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760773 -> 1789760774 00002000:00010000:0.0:1789760774.634023:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760773 -> 1789760774 00010000:00010000:0.0:1789760774.634040:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0681e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.634056:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0681e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.634066:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0681e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.634072:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760774.634079:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0681e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.634089:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0681e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.634112:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0681e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.634119:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0681e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.634240:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493817984:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37291us (37796us total) trans 4294973827 rc 0/0 00000100:00100000:0.0:1789760774.634254:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6538 00000100:00100000:0.0:1789760774.634267:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784894577871 time 784937954399 state 0x2 ref 0 00000100:00100000:1.0:1789760774.637530:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760774.637635:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493818112 00000100:00100000:3.0:1789760774.637647:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493818112 00000100:00100000:3.0:1789760774.637726:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784894577871 time 784937954399 state 0x0 ref 1 00000100:00100000:3.0:1789760774.637746:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784938214815 time 784937954399 state 0x1 ref 1 00000100:00100000:3.0:1789760774.637786:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784938214815 time 784978821225 state 0x0 ref 1 00000100:00100000:3.0:1789760774.637800:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+6:2369:x1876699493818112:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760774.637865:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26660864, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd007 00010000:00010000:3.0:1789760774.637876:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06825 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.637897:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=784s v=(0 0) 00010000:00010000:3.0:1789760774.637903:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.637911:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06825 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.637928:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06825 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760774.670042:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26660864 -> 26664960 00010000:00010000:1.0:1789760774.670073:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06825 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.670089:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06825 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.670099:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06825 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.670108:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760774.670113:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06825 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.670124:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06825 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.670153:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06825 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.670161:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06825 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.670916:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493818112:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33109us (33393us total) trans 4294973828 rc 0/0 00000100:00100000:1.0:1789760774.670929:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6539 00000100:00100000:1.0:1789760774.670942:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784938214815 time 784978821225 state 0x2 ref 0 00000001:00080000:1.0:1789760774.671037:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973828 is committed 00000100:00100000:1.0:1789760774.674671:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760774.675020:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493818240 00000100:00100000:3.0:1789760774.675034:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493818240 00000100:00100000:3.0:1789760774.675118:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784938214815 time 784978821225 state 0x0 ref 1 00000100:00100000:3.0:1789760774.675137:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784979081641 time 784978821225 state 0x1 ref 1 00000100:00100000:3.0:1789760774.675166:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784979081641 time 785016201297 state 0x0 ref 1 00000100:00100000:3.0:1789760774.675180:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493818240:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760774.675259:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26664960, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd007 00010000:00010000:3.0:1789760774.675270:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0682c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.675355:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=785s v=(0 0) 00010000:00010000:3.0:1789760774.675361:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.675368:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0682c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.675386:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0682c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.714147:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760774.717098:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973829 is committed 00002000:00010000:3.0:1789760774.717212:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26664960 -> 26669056 00010000:00010000:3.0:1789760774.717224:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0682c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.717248:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0682c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.717257:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0682c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.717262:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760774.717265:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0682c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.717271:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0682c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.717349:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0682c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.717357:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0682c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.717468:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493818240:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42286us (42822us total) trans 4294973829 rc 0/0 00000100:00100000:3.0:1789760774.717479:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6540 00000100:00100000:3.0:1789760774.717490:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784979081641 time 785016201297 state 0x2 ref 0 00000100:00100000:0.0:1789760774.721460:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760774.721594:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493818368 00000100:00100000:3.0:1789760774.721603:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493818368 00000100:00100000:3.0:1789760774.721769:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 784979081641 time 785016201297 state 0x0 ref 1 00000100:00100000:3.0:1789760774.721783:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785016461713 time 785016201297 state 0x1 ref 1 00000100:00100000:3.0:1789760774.721809:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785016461713 time 785062844933 state 0x0 ref 1 00000100:00100000:3.0:1789760774.721821:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493818368:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760774.721869:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26669056, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd007 00010000:00010000:3.0:1789760774.721930:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06833 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.721946:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=785s v=(0 0) 00010000:00010000:3.0:1789760774.721951:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.721957:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06833 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.722093:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06833 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.764227:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973830 is committed 00002000:00010000:1.0:1789760774.764420:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26669056 -> 26673152 00010000:00010000:1.0:1789760774.764439:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06833 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.764453:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06833 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.764461:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06833 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.764468:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760774.764473:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06833 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.764482:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06833 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.764503:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06833 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.764510:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06833 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.765208:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493818368:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42785us (43154us total) trans 4294973830 rc 0/0 00000100:00100000:1.0:1789760774.765223:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6541 00000100:00100000:1.0:1789760774.765241:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785016461713 time 785062844933 state 0x2 ref 0 00000100:00100000:1.0:1789760774.769713:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760774.769792:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493818496 00000100:00100000:0.0:1789760774.769802:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493818496 00000100:00100000:0.0:1789760774.769872:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785016461713 time 785062844933 state 0x0 ref 1 00000100:00100000:0.0:1789760774.769889:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785063105349 time 785062844933 state 0x1 ref 1 00000100:00100000:0.0:1789760774.769919:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785063105349 time 785110954601 state 0x0 ref 1 00000100:00100000:0.0:1789760774.769931:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493818496:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760774.770018:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26673152, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd007 00010000:00010000:0.0:1789760774.770028:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0683a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.770047:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=785s v=(0 0) 00010000:00010000:0.0:1789760774.770053:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.770060:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0683a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.770074:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0683a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.815307:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973831 is committed 00002000:00010000:3.0:1789760774.815479:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26673152 -> 26677248 00010000:00010000:3.0:1789760774.815494:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0683a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.815506:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0683a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.815515:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0683a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.815521:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760774.815526:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0683a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.815535:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0683a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.815553:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0683a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.815561:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e0683a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.815651:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493818496:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45716us (45942us total) trans 4294973831 rc 0/0 00000100:00100000:3.0:1789760774.815661:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6542 00000100:00100000:3.0:1789760774.815672:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785063105349 time 785110954601 state 0x2 ref 0 00000100:00100000:2.0:1789760774.821215:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760774.821301:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493818624 00000100:00100000:1.0:1789760774.821311:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493818624 00000100:00100000:1.0:1789760774.821365:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785063105349 time 785110954601 state 0x0 ref 1 00000100:00100000:1.0:1789760774.821381:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785111215017 time 785110954601 state 0x1 ref 1 00000100:00100000:1.0:1789760774.821412:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785111215017 time 785162447214 state 0x0 ref 1 00000100:00100000:1.0:1789760774.821425:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493818624:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760774.821473:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26677248, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd007 00010000:00010000:1.0:1789760774.821482:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06841 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.821497:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=785s v=(0 0) 00010000:00010000:1.0:1789760774.821502:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.821508:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06841 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.821522:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06841 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.861469:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973832 is committed 00002000:00010000:0.0:1789760774.861626:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26677248 -> 26681344 00010000:00010000:0.0:1789760774.861642:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06841 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.861655:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06841 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.861665:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06841 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.861673:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760774.861678:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06841 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.861693:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06841 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.861713:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06841 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.861721:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06841 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.861810:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493818624:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40384us (40599us total) trans 4294973832 rc 0/0 00000100:00100000:0.0:1789760774.861821:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6543 00000100:00100000:0.0:1789760774.861830:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785111215017 time 785162447214 state 0x2 ref 0 00000100:00100000:0.0:1789760774.870703:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760774.871044:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493818752 00000100:00100000:3.0:1789760774.871054:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493818752 00000100:00100000:3.0:1789760774.871108:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785111215017 time 785162447214 state 0x0 ref 1 00000100:00100000:3.0:1789760774.871123:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785162707630 time 785162447214 state 0x1 ref 1 00000100:00100000:3.0:1789760774.871151:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785162707630 time 785212187290 state 0x0 ref 1 00000100:00100000:3.0:1789760774.871163:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493818752:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760774.871226:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26681344, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd007 00010000:00010000:3.0:1789760774.871242:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06848 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.871261:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=785s v=(0 0) 00010000:00010000:3.0:1789760774.871266:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.871271:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06848 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.871290:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06848 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.917702:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973833 is committed 00002000:00010000:2.0:1789760774.917780:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26681344 -> 26685440 00010000:00010000:2.0:1789760774.917796:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06848 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.917809:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06848 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.917817:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06848 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.917824:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760774.917828:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06848 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.917837:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06848 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.917862:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06848 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.917932:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06848 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.918581:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493818752:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47414us (47884us total) trans 4294973833 rc 0/0 00000100:00100000:2.0:1789760774.918594:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6544 00000100:00100000:2.0:1789760774.918605:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785162707630 time 785212187290 state 0x2 ref 0 00000100:00100000:1.0:1789760774.922051:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760774.922185:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493818880 00000100:00100000:3.0:1789760774.922197:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493818880 00000100:00100000:3.0:1789760774.922271:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785162707630 time 785212187290 state 0x0 ref 1 00000100:00100000:3.0:1789760774.922391:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785212447706 time 785212187290 state 0x1 ref 1 00000100:00100000:3.0:1789760774.922417:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785212447706 time 785263452859 state 0x0 ref 1 00000100:00100000:3.0:1789760774.922431:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493818880:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760774.922485:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26685440, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd007 00010000:00010000:3.0:1789760774.922496:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0684f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.922511:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=785s v=(0 0) 00010000:00010000:3.0:1789760774.922516:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.922522:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0684f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.922536:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0684f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760774.956951:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26685440 -> 26689536 00010000:00010000:1.0:1789760774.957003:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0684f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.957016:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0684f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.957025:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0684f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.957033:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760774.957038:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0684f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.957047:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0684f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.957066:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0684f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.957074:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e0684f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.957277:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493818880:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34845us (35232us total) trans 4294973834 rc 0/0 00000100:00100000:1.0:1789760774.957360:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6545 00000100:00100000:1.0:1789760774.957370:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785212447706 time 785263452859 state 0x2 ref 0 00000001:00080000:2.0:1789760774.957832:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973834 is committed 00000100:00100000:1.0:1789760774.962838:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760774.962949:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493819008 00000100:00100000:2.0:1789760774.962989:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493819008 00000100:00100000:2.0:1789760774.963047:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785212447706 time 785263452859 state 0x0 ref 1 00000100:00100000:2.0:1789760774.963061:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785263713275 time 785263452859 state 0x1 ref 1 00000100:00100000:2.0:1789760774.963094:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785263713275 time 785304129158 state 0x0 ref 1 00000100:00100000:2.0:1789760774.963107:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493819008:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760774.963215:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26689536, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd007 00010000:00010000:2.0:1789760774.963224:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06856 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.963254:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=785s v=(0 0) 00010000:00010000:2.0:1789760774.963259:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.963265:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06856 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.963279:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06856 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.987903:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973835 is committed 00002000:00010000:2.0:1789760774.988045:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26689536 -> 26693632 00010000:00010000:2.0:1789760774.988061:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06856 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.988074:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06856 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.988083:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06856 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.988090:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760774.988095:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06856 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.988104:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06856 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.988123:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06856 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.988130:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06856 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.988222:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493819008:12345-192.168.203.60@tcp:4:dd.0 Request processed in 25113us (25390us total) trans 4294973835 rc 0/0 00000100:00100000:2.0:1789760774.989586:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6546 00000100:00100000:2.0:1789760774.989597:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785263713275 time 785304129158 state 0x2 ref 0 00000100:00100000:1.0:1789760774.994478:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760774.994580:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493819136 00000100:00100000:0.0:1789760774.994591:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493819136 00000100:00100000:0.0:1789760774.994650:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785263713275 time 785304129158 state 0x0 ref 1 00000100:00100000:0.0:1789760774.994667:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785304389574 time 785304129158 state 0x1 ref 1 00000100:00100000:0.0:1789760774.994857:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785304389574 time 785335890722 state 0x0 ref 1 00000100:00100000:0.0:1789760774.994871:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493819136:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760774.994923:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26693632, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd007 00010000:00010000:0.0:1789760774.994933:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0685d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.994948:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=785s v=(0 0) 00010000:00010000:0.0:1789760774.994953:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.994986:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0685d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.995002:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0685d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760775.030868:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26693632 -> 26697728 00010000:00010000:1.0:1789760775.030887:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0685d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.030899:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0685d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.030906:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0685d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.030912:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760775.030916:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0685d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.030926:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0685d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.030945:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0685d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.030976:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0685d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.031073:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493819136:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36200us (36601us total) trans 4294973836 rc 0/0 00000100:00100000:1.0:1789760775.031084:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6547 00000100:00100000:1.0:1789760775.031094:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785304389574 time 785335890722 state 0x2 ref 0 00000001:00080000:2.0:1789760775.032002:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973836 is committed 00000100:00100000:1.0:1789760775.034463:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760775.034514:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4528380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3303:1876699519259520:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760775.034564:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760775.035073:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493819264 00000100:00100000:3.0:1789760775.035081:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493819264 00000100:00100000:3.0:1789760775.035142:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785304389574 time 785335890722 state 0x0 ref 1 00000100:00100000:3.0:1789760775.035156:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785336151138 time 785335890722 state 0x1 ref 1 00000100:00100000:3.0:1789760775.035184:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785336151138 time 785376219946 state 0x0 ref 1 00000100:00100000:3.0:1789760775.035195:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493819264:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760775.035258:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26697728, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd007 00010000:00010000:3.0:1789760775.035267:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06864 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.035282:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=785s v=(0 0) 00010000:00010000:3.0:1789760775.035287:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.035293:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06864 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.035306:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06864 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760775.035458:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760775.036248:0:6691:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519259520 00000100:00100000:2.0:1789760775.036255:0:6691:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519259520 00000100:00100000:2.0:1789760775.036364:0:6691:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 213 00000100:00100000:2.0:1789760775.036374:0:6691:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc14e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876699519259520:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760775.036401:0:6691:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760775.036474:0:6691:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc14e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876699519259520:12345-0@lo:13:osp-pre-1-0.0 Request processed in 102us (1908us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760775.036481:0:6691:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 213 00000100:00100000:1.0:1789760775.036569:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4528380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3303:1876699519259520:0@lo:13:osp-pre-1-0.0 00002000:00010000:2.0:1789760775.065394:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26697728 -> 26701824 00000001:00080000:1.0:1789760775.065396:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973837 is committed 00010000:00010000:2.0:1789760775.065412:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06864 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.065425:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06864 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.065433:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06864 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.065440:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760775.065444:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06864 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.065454:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06864 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.065476:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06864 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.065484:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06864 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.065586:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493819264:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30387us (31122us total) trans 4294973837 rc 0/0 00000100:00100000:2.0:1789760775.065597:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6548 00000100:00100000:2.0:1789760775.065714:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785336151138 time 785376219946 state 0x2 ref 0 00000100:00100000:0.0:1789760775.071936:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760775.072208:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493819392 00000100:00100000:1.0:1789760775.072218:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493819392 00000100:00100000:1.0:1789760775.072294:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785336151138 time 785376219946 state 0x0 ref 1 00000100:00100000:1.0:1789760775.072308:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785376480362 time 785376219946 state 0x1 ref 1 00000100:00100000:1.0:1789760775.072341:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785376480362 time 785413377166 state 0x0 ref 1 00000100:00100000:1.0:1789760775.072354:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493819392:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760775.072401:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26701824, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd007 00010000:00010000:1.0:1789760775.072410:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0686b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.072430:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=785s v=(0 0) 00010000:00010000:1.0:1789760775.072435:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.072441:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0686b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.072456:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0686b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.103203:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973838 is committed 00002000:00010000:2.0:1789760775.103350:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26701824 -> 26705920 00010000:00010000:2.0:1789760775.103364:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0686b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.103373:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0686b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.103378:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0686b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.103383:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760775.103386:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0686b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.103395:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0686b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.103410:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0686b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.103414:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e0686b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26701824->26705919] (req 26701824->26705919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.103524:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493819392:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31167us (31590us total) trans 4294973838 rc 0/0 00000100:00100000:2.0:1789760775.103532:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6549 00000100:00100000:2.0:1789760775.103540:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785376480362 time 785413377166 state 0x2 ref 0 00000100:00100000:1.0:1789760775.106722:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760775.106819:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493819520 00000100:00100000:2.0:1789760775.106828:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493819520 00000100:00100000:2.0:1789760775.106879:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785376480362 time 785413377166 state 0x0 ref 1 00000100:00100000:2.0:1789760775.106893:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785413637582 time 785413377166 state 0x1 ref 1 00000100:00100000:2.0:1789760775.107083:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785413637582 time 785448117112 state 0x0 ref 1 00000100:00100000:2.0:1789760775.107096:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d8b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493819520:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760775.107143:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26705920, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd007 00010000:00010000:2.0:1789760775.107152:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06872 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.107169:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=785s v=(0 0) 00010000:00010000:2.0:1789760775.107174:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.107180:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06872 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.107193:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06872 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.140742:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973839 is committed 00002000:00010000:1.0:1789760775.140847:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26705920 -> 26710016 00010000:00010000:1.0:1789760775.140861:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06872 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.140873:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06872 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.140882:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06872 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.140888:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760775.140893:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06872 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.140902:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06872 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.140925:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06872 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.140932:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06872 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26705920->26710015] (req 26705920->26710015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.141060:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d8b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493819520:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33962us (34343us total) trans 4294973839 rc 0/0 00000100:00100000:1.0:1789760775.141071:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6550 00000100:00100000:1.0:1789760775.141080:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785413637582 time 785448117112 state 0x2 ref 0 00000100:00100000:1.0:1789760775.147254:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760775.147452:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493819648 00000100:00100000:0.0:1789760775.147462:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493819648 00000100:00100000:0.0:1789760775.147516:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785413637582 time 785448117112 state 0x0 ref 1 00000100:00100000:0.0:1789760775.147531:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785448377528 time 785448117112 state 0x1 ref 1 00000100:00100000:0.0:1789760775.147561:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785448377528 time 785488596848 state 0x0 ref 1 00000100:00100000:0.0:1789760775.147574:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493819648:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760775.147623:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26710016, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd007 00010000:00010000:0.0:1789760775.147631:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06879 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.147645:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=785s v=(0 0) 00010000:00010000:0.0:1789760775.147650:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.147655:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06879 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.147667:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06879 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.175204:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973840 is committed 00002000:00010000:2.0:1789760775.175308:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26710016 -> 26714112 00010000:00010000:2.0:1789760775.175323:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06879 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.175337:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06879 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.175346:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06879 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.175353:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760775.175359:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06879 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.175374:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06879 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.175399:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06879 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.175404:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06879 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26710016->26714111] (req 26710016->26714111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.175505:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493819648:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27930us (28273us total) trans 4294973840 rc 0/0 00000100:00100000:2.0:1789760775.175514:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6551 00000100:00100000:2.0:1789760775.175523:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785448377528 time 785488596848 state 0x2 ref 0 00000100:00100000:1.0:1789760775.179411:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760775.179707:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493819776 00000100:00100000:3.0:1789760775.179717:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493819776 00000100:00100000:3.0:1789760775.179769:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785448377528 time 785488596848 state 0x0 ref 1 00000100:00100000:3.0:1789760775.179784:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785488857264 time 785488596848 state 0x1 ref 1 00000100:00100000:3.0:1789760775.179812:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785488857264 time 785520848289 state 0x0 ref 1 00000100:00100000:3.0:1789760775.179824:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c89fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493819776:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760775.179871:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26714112, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd007 00010000:00010000:3.0:1789760775.179879:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06880 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.179895:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=785s v=(0 0) 00010000:00010000:3.0:1789760775.179900:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.179907:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06880 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.179921:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06880 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.209182:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973841 is committed 00002000:00010000:3.0:1789760775.209446:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26714112 -> 26718208 00010000:00010000:3.0:1789760775.209462:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06880 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.209475:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06880 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.209484:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06880 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.209492:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760775.209497:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06880 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.209506:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06880 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.209525:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06880 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.209531:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06880 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26714112->26718207] (req 26714112->26718207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.209621:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c89fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493819776:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29794us (30216us total) trans 4294973841 rc 0/0 00000100:00100000:3.0:1789760775.209632:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6552 00000100:00100000:3.0:1789760775.209642:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785488857264 time 785520848289 state 0x2 ref 0 00000100:00100000:1.0:1789760775.215973:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760775.216120:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493819904 00000100:00100000:2.0:1789760775.216127:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493819904 00000100:00100000:2.0:1789760775.216171:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785488857264 time 785520848289 state 0x0 ref 1 00000100:00100000:2.0:1789760775.216182:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785521108705 time 785520848289 state 0x1 ref 1 00000100:00100000:2.0:1789760775.216209:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785521108705 time 785557245129 state 0x0 ref 1 00000100:00100000:2.0:1789760775.216221:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fadb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493819904:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760775.216278:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26718208, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd007 00010000:00010000:2.0:1789760775.216286:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06887 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.216306:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=785s v=(0 0) 00010000:00010000:2.0:1789760775.216311:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.216317:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06887 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.216331:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06887 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760775.247868:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26718208 -> 26722304 00010000:00010000:1.0:1789760775.247889:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06887 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.247902:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06887 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.247910:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06887 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.247918:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760775.247923:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06887 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.247932:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06887 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.247951:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06887 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.247978:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06887 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26718208->26722303] (req 26718208->26722303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.248073:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fadb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493819904:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31847us (32117us total) trans 4294973842 rc 0/0 00000100:00100000:1.0:1789760775.248084:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6553 00000100:00100000:1.0:1789760775.248093:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785521108705 time 785557245129 state 0x2 ref 0 00000001:00080000:3.0:1789760775.248667:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973842 is committed 00000100:00100000:1.0:1789760775.256643:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760775.258726:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493820032 00000100:00100000:2.0:1789760775.258740:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493820032 00000100:00100000:2.0:1789760775.258803:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785521108705 time 785557245129 state 0x0 ref 1 00000100:00100000:2.0:1789760775.258818:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785557505545 time 785557245129 state 0x1 ref 1 00000100:00100000:2.0:1789760775.258854:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785557505545 time 785599889655 state 0x0 ref 1 00000100:00100000:2.0:1789760775.258867:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493820032:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760775.258913:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26722304, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd007 00010000:00010000:2.0:1789760775.258921:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0688e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.258936:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=785s v=(0 0) 00010000:00010000:2.0:1789760775.258941:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.258947:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0688e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.259026:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0688e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.293194:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973843 is committed 00002000:00010000:2.0:1789760775.293319:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26722304 -> 26726400 00010000:00010000:2.0:1789760775.293334:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0688e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.293347:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0688e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.293355:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0688e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.293362:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760775.293367:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0688e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.293376:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0688e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.293395:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0688e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.293404:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0688e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26722304->26726399] (req 26722304->26726399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.293504:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493820032:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34634us (36865us total) trans 4294973843 rc 0/0 00000100:00100000:2.0:1789760775.293514:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6554 00000100:00100000:2.0:1789760775.293523:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785557505545 time 785599889655 state 0x2 ref 0 00000100:00100000:0.0:1789760775.300316:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760775.300467:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493820160 00000100:00100000:2.0:1789760775.300476:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493820160 00000100:00100000:2.0:1789760775.300532:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785557505545 time 785599889655 state 0x0 ref 1 00000100:00100000:2.0:1789760775.300547:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785600150071 time 785599889655 state 0x1 ref 1 00000100:00100000:2.0:1789760775.300582:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785600150071 time 785641618206 state 0x0 ref 1 00000100:00100000:2.0:1789760775.300595:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493820160:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760775.300642:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26726400, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd007 00010000:00010000:2.0:1789760775.300651:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06895 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.300667:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=785s v=(0 0) 00010000:00010000:2.0:1789760775.300671:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.300677:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06895 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.300691:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06895 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.329990:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973844 is committed 00002000:00010000:2.0:1789760775.330196:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26726400 -> 26730496 00010000:00010000:2.0:1789760775.330210:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06895 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.330222:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06895 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.330240:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06895 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.330247:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760775.330252:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06895 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.330621:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06895 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.330644:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06895 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.330652:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06895 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26726400->26730495] (req 26726400->26730495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.331374:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493820160:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30192us (30478us total) trans 4294973844 rc 0/0 00000100:00100000:2.0:1789760775.331391:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6555 00000100:00100000:2.0:1789760775.331400:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785600150071 time 785641618206 state 0x2 ref 0 00000100:00100000:0.0:1789760775.338636:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760775.339142:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493820288 00000100:00100000:3.0:1789760775.339153:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493820288 00000100:00100000:2.0:1789760775.339531:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785600150071 time 785641618206 state 0x0 ref 1 00000100:00100000:2.0:1789760775.339547:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785641878622 time 785641618206 state 0x1 ref 1 00000100:00100000:2.0:1789760775.339619:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785641878622 time 785680653026 state 0x0 ref 1 00000100:00100000:2.0:1789760775.339633:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493820288:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760775.339684:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26730496, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd007 00010000:00010000:2.0:1789760775.339692:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0689c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.339721:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=785s v=(0 0) 00010000:00010000:2.0:1789760775.339726:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.339731:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0689c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.339746:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0689c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760775.380082:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26730496 -> 26734592 00010000:00010000:3.0:1789760775.380103:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0689c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.380116:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0689c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.380126:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0689c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.380133:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760775.380138:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0689c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.380148:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0689c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.380172:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0689c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.380180:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0689c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26730496->26734591] (req 26730496->26734591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.380473:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493820288:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40835us (41840us total) trans 4294973845 rc 0/0 00000100:00100000:3.0:1789760775.380486:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6556 00000100:00100000:3.0:1789760775.380498:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785641878622 time 785680653026 state 0x2 ref 0 00000001:00080000:0.0:1789760775.380600:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973845 is committed 00000100:00100000:1.0:1789760775.383826:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760775.384673:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493820416 00000100:00100000:2.0:1789760775.384683:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493820416 00000100:00100000:2.0:1789760775.384754:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785641878622 time 785680653026 state 0x0 ref 1 00000100:00100000:2.0:1789760775.384772:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785680913442 time 785680653026 state 0x1 ref 1 00000100:00100000:2.0:1789760775.384807:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785680913442 time 785725841992 state 0x0 ref 1 00000100:00100000:2.0:1789760775.384820:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493820416:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760775.384870:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26734592, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd007 00010000:00010000:2.0:1789760775.384878:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e068a3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.384894:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=785s v=(0 0) 00010000:00010000:2.0:1789760775.384898:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.384904:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e068a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.384917:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e068a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.412080:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973846 is committed 00002000:00010000:2.0:1789760775.412206:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26734592 -> 26738688 00010000:00010000:2.0:1789760775.412222:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e068a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.412245:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e068a3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.412254:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e068a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.412266:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760775.412271:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e068a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.412344:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e068a3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.412364:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e068a3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.412372:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e068a3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26734592->26738687] (req 26734592->26738687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.412512:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493820416:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27690us (28690us total) trans 4294973846 rc 0/0 00000100:00100000:2.0:1789760775.412523:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6557 00000100:00100000:2.0:1789760775.412532:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785680913442 time 785725841992 state 0x2 ref 0 00000100:00100000:2.0:1789760775.416422:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760775.416521:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493820544 00000100:00100000:1.0:1789760775.416530:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493820544 00000100:00100000:1.0:1789760775.416586:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785680913442 time 785725841992 state 0x0 ref 1 00000100:00100000:1.0:1789760775.416601:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785726102408 time 785725841992 state 0x1 ref 1 00000100:00100000:1.0:1789760775.416673:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785726102408 time 785757707906 state 0x0 ref 1 00000100:00100000:1.0:1789760775.416686:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493820544:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760775.416746:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26738688, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd007 00010000:00010000:1.0:1789760775.416756:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e068aa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.416771:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=785s v=(0 0) 00010000:00010000:1.0:1789760775.416775:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.416782:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e068aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.416796:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e068aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760775.444702:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26738688 -> 26742784 00010000:00010000:3.0:1789760775.444799:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e068aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.444812:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e068aa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.444822:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e068aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.444830:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760775.444834:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e068aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.444848:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e068aa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.444868:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e068aa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.444876:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e068aa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26738688->26742783] (req 26738688->26742783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.445007:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493820544:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28319us (28590us total) trans 4294973847 rc 0/0 00000100:00100000:3.0:1789760775.445018:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6558 00000100:00100000:3.0:1789760775.445028:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785726102408 time 785757707906 state 0x2 ref 0 00000100:00100000:2.0:1789760775.448130:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760775.448220:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493820672 00000100:00100000:3.0:1789760775.448240:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493820672 00000100:00100000:3.0:1789760775.448308:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785726102408 time 785757707906 state 0x0 ref 1 00000100:00100000:3.0:1789760775.448325:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785757968322 time 785757707906 state 0x1 ref 1 00000100:00100000:3.0:1789760775.448356:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785757968322 time 785789391468 state 0x0 ref 1 00000100:00100000:3.0:1789760775.448370:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493820672:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760775.448420:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26742784, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd007 00010000:00010000:3.0:1789760775.448429:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e068b1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.448445:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=785s v=(0 0) 00010000:00010000:3.0:1789760775.448449:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.448456:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e068b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.448470:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e068b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.449371:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973847 is committed 00002000:00010000:0.0:1789760775.483187:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26742784 -> 26746880 00010000:00010000:0.0:1789760775.483214:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e068b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.483237:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e068b1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.483246:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e068b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.483254:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760775.483259:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e068b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.483269:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e068b1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.483300:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e068b1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.483308:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e068b1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26742784->26746879] (req 26742784->26746879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.483407:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493820672:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35035us (35282us total) trans 4294973848 rc 0/0 00000100:00100000:0.0:1789760775.483418:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6559 00000100:00100000:0.0:1789760775.483428:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785757968322 time 785789391468 state 0x2 ref 0 00000100:00100000:1.0:1789760775.487328:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760775.487491:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493820800 00000100:00100000:3.0:1789760775.487502:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493820800 00000100:00100000:3.0:1789760775.487556:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785757968322 time 785789391468 state 0x0 ref 1 00000100:00100000:3.0:1789760775.487571:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785789651884 time 785789391468 state 0x1 ref 1 00000100:00100000:3.0:1789760775.487603:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785789651884 time 785828638260 state 0x0 ref 1 00000100:00100000:3.0:1789760775.487616:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493820800:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760775.487666:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26746880, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd007 00010000:00010000:3.0:1789760775.487674:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e068b8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.487691:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=785s v=(0 0) 00010000:00010000:3.0:1789760775.487695:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.487701:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e068b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.487733:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e068b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.489543:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973848 is committed 00000001:00080000:0.0:1789760775.516071:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973849 is committed 00002000:00010000:2.0:1789760775.516207:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26746880 -> 26750976 00010000:00010000:2.0:1789760775.516231:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e068b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.516250:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e068b8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.516260:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e068b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.516267:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760775.516272:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e068b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.516350:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e068b8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.516370:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e068b8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.516377:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e068b8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26746880->26750975] (req 26746880->26750975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.516511:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493820800:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28893us (29189us total) trans 4294973849 rc 0/0 00000100:00100000:2.0:1789760775.516525:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6560 00000100:00100000:2.0:1789760775.516533:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785789651884 time 785828638260 state 0x2 ref 0 00000100:00100000:2.0:1789760775.519874:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760775.520024:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493820928 00000100:00100000:1.0:1789760775.520034:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493820928 00000100:00100000:1.0:1789760775.520375:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785789651884 time 785828638260 state 0x0 ref 1 00000100:00100000:1.0:1789760775.520395:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785828898676 time 785828638260 state 0x1 ref 1 00000100:00100000:1.0:1789760775.520461:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785828898676 time 785861496114 state 0x0 ref 1 00000100:00100000:1.0:1789760775.520475:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493820928:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760775.520525:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26750976, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd007 00010000:00010000:1.0:1789760775.520534:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e068bf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.520554:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=785s v=(0 0) 00010000:00010000:1.0:1789760775.520558:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.520564:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e068bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.520578:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e068bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.547522:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973850 is committed 00002000:00010000:1.0:1789760775.547672:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26750976 -> 26755072 00010000:00010000:1.0:1789760775.547688:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e068bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.547700:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e068bf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.547784:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e068bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.547792:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760775.547796:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e068bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.547806:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e068bf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.547826:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e068bf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.547834:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e068bf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26750976->26755071] (req 26750976->26755071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.547998:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493820928:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27489us (28094us total) trans 4294973850 rc 0/0 00000100:00100000:1.0:1789760775.548009:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6561 00000100:00100000:1.0:1789760775.548019:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785828898676 time 785861496114 state 0x2 ref 0 00000100:00100000:1.0:1789760775.552078:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760775.552162:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493821056 00000100:00100000:3.0:1789760775.552171:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493821056 00000100:00100000:3.0:1789760775.552235:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785828898676 time 785861496114 state 0x0 ref 1 00000100:00100000:3.0:1789760775.552251:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785861756530 time 785861496114 state 0x1 ref 1 00000100:00100000:3.0:1789760775.552325:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785861756530 time 785893360055 state 0x0 ref 1 00000100:00100000:3.0:1789760775.552338:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493821056:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760775.552386:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26755072, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd007 00010000:00010000:3.0:1789760775.552394:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e068c6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.552410:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=785s v=(0 0) 00010000:00010000:3.0:1789760775.552415:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.552421:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e068c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.552436:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e068c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760775.579649:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26755072 -> 26759168 00010000:00010000:3.0:1789760775.579669:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e068c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.579682:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e068c6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.579692:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e068c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.579699:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760775.579715:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e068c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.579725:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e068c6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.579744:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e068c6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.579752:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e068c6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26755072->26759167] (req 26755072->26759167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.579832:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493821056:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27491us (27759us total) trans 4294973851 rc 0/0 00000100:00100000:3.0:1789760775.579842:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6562 00000100:00100000:3.0:1789760775.579851:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785861756530 time 785893360055 state 0x2 ref 0 00000001:00080000:0.0:1789760775.581480:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973851 is committed 00000100:00100000:2.0:1789760775.583560:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760775.583632:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493821184 00000100:00100000:0.0:1789760775.583642:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493821184 00000100:00100000:0.0:1789760775.583693:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785861756530 time 785893360055 state 0x0 ref 1 00000100:00100000:0.0:1789760775.583762:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785893620471 time 785893360055 state 0x1 ref 1 00000100:00100000:0.0:1789760775.583789:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785893620471 time 785924825730 state 0x0 ref 1 00000100:00100000:0.0:1789760775.583801:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8555880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493821184:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760775.583844:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26759168, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xd007 00010000:00010000:0.0:1789760775.583853:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e068cd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.583868:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=785s v=(0 0) 00010000:00010000:0.0:1789760775.583873:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.583878:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e068cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.583891:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e068cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.606999:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973852 is committed 00002000:00010000:3.0:1789760775.607110:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26759168 -> 26763264 00002000:00010000:3.0:1789760775.607116:0:13038:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760774 -> 1789760775 00002000:00010000:3.0:1789760775.607119:0:13038:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760774 -> 1789760775 00010000:00010000:3.0:1789760775.607135:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e068cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.607150:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e068cd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.607159:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e068cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.607166:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760775.607171:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e068cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.607181:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e068cd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.607201:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e068cd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.607209:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e068cd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26759168->26763263] (req 26759168->26763263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.607468:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8555880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493821184:12345-192.168.203.60@tcp:4:dd.0 Request processed in 23662us (23911us total) trans 4294973852 rc 0/0 00000100:00100000:3.0:1789760775.607479:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6563 00000100:00100000:3.0:1789760775.607489:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785893620471 time 785924825730 state 0x2 ref 0 00000100:00100000:3.0:1789760775.611526:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760775.612194:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493821312 00000100:00100000:2.0:1789760775.612204:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493821312 00000100:00100000:2.0:1789760775.612271:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785893620471 time 785924825730 state 0x0 ref 1 00000100:00100000:2.0:1789760775.612342:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785925086146 time 785924825730 state 0x1 ref 1 00000100:00100000:2.0:1789760775.612400:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785925086146 time 785953435212 state 0x0 ref 1 00000100:00100000:2.0:1789760775.612414:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493821312:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760775.612461:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26763264, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd007 00010000:00010000:2.0:1789760775.612470:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e068d4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.612487:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=785s v=(0 0) 00010000:00010000:2.0:1789760775.612491:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.612498:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e068d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.612512:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e068d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760775.647211:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26763264 -> 26767360 00010000:00010000:3.0:1789760775.647241:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e068d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.647252:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e068d4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.647260:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e068d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.647267:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760775.647271:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e068d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.647278:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e068d4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.647295:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e068d4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.647302:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e068d4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26763264->26767359] (req 26763264->26767359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.647397:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493821312:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34980us (35875us total) trans 4294973853 rc 0/0 00000100:00100000:3.0:1789760775.647407:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6564 00000100:00100000:3.0:1789760775.647421:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785925086146 time 785953435212 state 0x2 ref 0 00000001:00080000:2.0:1789760775.647429:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973853 is committed 00000100:00100000:1.0:1789760775.651818:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760775.652057:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493821440 00000100:00100000:2.0:1789760775.652067:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493821440 00000100:00100000:3.0:1789760775.652180:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785925086146 time 785953435212 state 0x0 ref 1 00000100:00100000:3.0:1789760775.652193:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785953695628 time 785953435212 state 0x1 ref 1 00000100:00100000:0.0:1789760775.652297:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785953695628 time 785993330075 state 0x0 ref 1 00000100:00100000:0.0:1789760775.652312:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493821440:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760775.652357:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26767360, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd007 00010000:00010000:0.0:1789760775.652365:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e068db lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.652381:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=785s v=(0 0) 00010000:00010000:0.0:1789760775.652385:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.652390:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e068db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.652403:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e068db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.683945:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973854 is committed 00002000:00010000:0.0:1789760775.684093:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26767360 -> 26771456 00010000:00010000:0.0:1789760775.684110:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e068db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.684128:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e068db lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.684137:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e068db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.684145:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760775.684151:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e068db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.684161:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e068db lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.684182:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e068db lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.684190:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e068db lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26767360->26771455] (req 26767360->26771455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.684342:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493821440:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32028us (32533us total) trans 4294973854 rc 0/0 00000100:00100000:0.0:1789760775.684353:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6565 00000100:00100000:0.0:1789760775.684362:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785953695628 time 785993330075 state 0x2 ref 0 00000100:00100000:1.0:1789760775.687161:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760775.687220:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493821568 00000100:00100000:3.0:1789760775.687243:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493821568 00000100:00100000:3.0:1789760775.687338:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785953695628 time 785993330075 state 0x0 ref 1 00000100:00100000:3.0:1789760775.687349:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785993590491 time 785993330075 state 0x1 ref 1 00000100:00100000:3.0:1789760775.687375:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785993590491 time 786028410205 state 0x0 ref 1 00000100:00100000:3.0:1789760775.687387:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493821568:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760775.687433:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26771456, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd007 00010000:00010000:3.0:1789760775.687441:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e068e2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.687457:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=786s v=(0 0) 00010000:00010000:3.0:1789760775.687461:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.687469:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e068e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.687483:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e068e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.721004:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973855 is committed 00002000:00010000:0.0:1789760775.721075:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26771456 -> 26775552 00010000:00010000:0.0:1789760775.721091:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e068e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.721104:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e068e2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.721113:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e068e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.721121:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760775.721125:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e068e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.721135:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e068e2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.721210:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e068e2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.721231:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e068e2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26771456->26775551] (req 26771456->26775551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.721493:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493821568:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34103us (34336us total) trans 4294973855 rc 0/0 00000100:00100000:0.0:1789760775.721505:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6566 00000100:00100000:0.0:1789760775.721515:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785993590491 time 786028410205 state 0x2 ref 0 00000100:00100000:3.0:1789760775.733076:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760775.733216:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493821696 00000100:00100000:1.0:1789760775.733233:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493821696 00000100:00100000:1.0:1789760775.733361:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 785993590491 time 786028410205 state 0x0 ref 1 00000100:00100000:1.0:1789760775.733378:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786028670621 time 786028410205 state 0x1 ref 1 00000100:00100000:1.0:1789760775.733410:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786028670621 time 786074445842 state 0x0 ref 1 00000100:00100000:1.0:1789760775.733424:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493821696:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760775.733473:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26775552, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd007 00010000:00010000:1.0:1789760775.733482:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e068e9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.733504:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=786s v=(0 0) 00010000:00010000:1.0:1789760775.733508:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.733514:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e068e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.733529:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e068e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.748277:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760775.786719:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973856 is committed 00002000:00010000:3.0:1789760775.786902:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26775552 -> 26779648 00010000:00010000:3.0:1789760775.786920:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e068e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.786933:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e068e9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.786942:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e068e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.786949:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760775.786954:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e068e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.786996:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e068e9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.787016:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e068e9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.787023:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e068e9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26775552->26779647] (req 26775552->26779647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.787121:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493821696:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53695us (54050us total) trans 4294973856 rc 0/0 00000100:00100000:3.0:1789760775.787131:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6567 00000100:00100000:3.0:1789760775.787141:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786028670621 time 786074445842 state 0x2 ref 0 00000100:00100000:2.0:1789760775.790070:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760775.790165:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493821824 00000100:00100000:3.0:1789760775.790174:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493821824 00000100:00100000:3.0:1789760775.790241:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786028670621 time 786074445842 state 0x0 ref 1 00000100:00100000:3.0:1789760775.790256:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786074706258 time 786074445842 state 0x1 ref 1 00000100:00100000:3.0:1789760775.790286:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786074706258 time 786131321108 state 0x0 ref 1 00000100:00100000:3.0:1789760775.790298:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493821824:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760775.790345:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26779648, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd007 00010000:00010000:3.0:1789760775.790354:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e068f0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.790369:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=786s v=(0 0) 00010000:00010000:3.0:1789760775.790374:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.790380:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e068f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.790394:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e068f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.821482:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973857 is committed 00002000:00010000:2.0:1789760775.821606:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26779648 -> 26783744 00010000:00010000:2.0:1789760775.821622:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e068f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.821633:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e068f0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.821641:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e068f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.821649:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760775.821654:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e068f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.821663:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e068f0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.821682:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e068f0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.821691:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e068f0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26779648->26783743] (req 26779648->26783743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.821953:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493821824:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31651us (31885us total) trans 4294973857 rc 0/0 00000100:00100000:2.0:1789760775.822015:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6568 00000100:00100000:2.0:1789760775.822026:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786074706258 time 786131321108 state 0x2 ref 0 00000100:00100000:2.0:1789760775.827154:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760775.827890:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493821952 00000100:00100000:0.0:1789760775.827901:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493821952 00000100:00100000:0.0:1789760775.828050:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786074706258 time 786131321108 state 0x0 ref 1 00000100:00100000:0.0:1789760775.828069:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786131581524 time 786131321108 state 0x1 ref 1 00000100:00100000:1.0:1789760775.828205:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786131581524 time 786169238396 state 0x0 ref 1 00000100:00100000:1.0:1789760775.828235:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493821952:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760775.828404:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26783744, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd007 00010000:00010000:1.0:1789760775.828414:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e068f7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.828430:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=786s v=(0 0) 00010000:00010000:1.0:1789760775.828435:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.828441:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e068f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.828454:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e068f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.882310:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973858 is committed 00002000:00010000:2.0:1789760775.882501:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26783744 -> 26787840 00010000:00010000:2.0:1789760775.882516:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e068f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.882529:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e068f7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.882539:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e068f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.882546:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760775.882550:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e068f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.882560:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e068f7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.882579:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e068f7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.882586:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e068f7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26783744->26787839] (req 26783744->26787839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.882676:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493821952:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54453us (55526us total) trans 4294973858 rc 0/0 00000100:00100000:2.0:1789760775.882690:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6569 00000100:00100000:2.0:1789760775.882700:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786131581524 time 786169238396 state 0x2 ref 0 00000100:00100000:3.0:1789760775.885644:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760775.885805:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493822080 00000100:00100000:1.0:1789760775.885816:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493822080 00000100:00100000:1.0:1789760775.885874:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786131581524 time 786169238396 state 0x0 ref 1 00000100:00100000:1.0:1789760775.885889:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786169498812 time 786169238396 state 0x1 ref 1 00000100:00100000:1.0:1789760775.885924:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786169498812 time 786226959798 state 0x0 ref 1 00000100:00100000:1.0:1789760775.885936:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493822080:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760775.886022:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26787840, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd007 00010000:00010000:1.0:1789760775.886030:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e068fe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.886046:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=786s v=(0 0) 00010000:00010000:1.0:1789760775.886051:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.886057:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e068fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.886071:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e068fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.926391:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973859 is committed 00002000:00010000:1.0:1789760775.926633:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26787840 -> 26791936 00010000:00010000:1.0:1789760775.926649:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e068fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.926662:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e068fe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.926671:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e068fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.926680:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760775.926684:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e068fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.926694:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e068fe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.926725:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e068fe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.926734:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e068fe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26787840->26791935] (req 26787840->26791935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.926865:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493822080:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40924us (41224us total) trans 4294973859 rc 0/0 00000100:00100000:1.0:1789760775.926876:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6570 00000100:00100000:1.0:1789760775.926886:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786169498812 time 786226959798 state 0x2 ref 0 00000100:00100000:1.0:1789760775.930026:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760775.930119:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493822208 00000100:00100000:3.0:1789760775.930128:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493822208 00000100:00100000:3.0:1789760775.930184:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786169498812 time 786226959798 state 0x0 ref 1 00000100:00100000:3.0:1789760775.930199:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786227220214 time 786226959798 state 0x1 ref 1 00000100:00100000:3.0:1789760775.930291:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786227220214 time 786271325404 state 0x0 ref 1 00000100:00100000:3.0:1789760775.930305:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493822208:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760775.930352:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26791936, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd007 00010000:00010000:3.0:1789760775.930361:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06905 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.930376:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=786s v=(0 0) 00010000:00010000:3.0:1789760775.930380:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.930386:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06905 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.930398:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06905 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.960817:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973860 is committed 00002000:00010000:3.0:1789760775.961011:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26791936 -> 26796032 00010000:00010000:3.0:1789760775.961026:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06905 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.961039:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06905 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.961048:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06905 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.961055:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760775.961059:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06905 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.961068:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06905 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.961089:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06905 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.961095:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06905 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26791936->26796031] (req 26791936->26796031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.961187:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493822208:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30880us (31168us total) trans 4294973860 rc 0/0 00000100:00100000:3.0:1789760775.961197:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6571 00000100:00100000:3.0:1789760775.961207:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786227220214 time 786271325404 state 0x2 ref 0 00000100:00100000:2.0:1789760775.965154:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760775.965487:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493822336 00000100:00100000:3.0:1789760775.965497:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493822336 00000100:00100000:3.0:1789760775.965556:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786227220214 time 786271325404 state 0x0 ref 1 00000100:00100000:3.0:1789760775.965571:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786271585820 time 786271325404 state 0x1 ref 1 00000100:00100000:3.0:1789760775.965689:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786271585820 time 786306723523 state 0x0 ref 1 00000100:00100000:3.0:1789760775.965714:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493822336:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760775.965764:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26796032, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd007 00010000:00010000:3.0:1789760775.965773:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0690c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.965788:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=786s v=(0 0) 00010000:00010000:3.0:1789760775.965792:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.965799:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0690c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.965812:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0690c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.997407:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973861 is committed 00002000:00010000:3.0:1789760775.997792:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26796032 -> 26800128 00010000:00010000:3.0:1789760775.997807:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0690c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.997820:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0690c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.997836:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0690c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.997843:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760775.997848:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0690c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.997857:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0690c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.997876:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0690c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.997883:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0690c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26796032->26800127] (req 26796032->26800127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.998032:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493822336:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32326us (32884us total) trans 4294973861 rc 0/0 00000100:00100000:3.0:1789760775.998043:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6572 00000100:00100000:3.0:1789760775.998052:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786271585820 time 786306723523 state 0x2 ref 0 00000100:00100000:2.0:1789760776.001254:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760776.001392:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493822464 00000100:00100000:1.0:1789760776.001402:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493822464 00000100:00100000:1.0:1789760776.001455:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786271585820 time 786306723523 state 0x0 ref 1 00000100:00100000:1.0:1789760776.001471:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786306983939 time 786306723523 state 0x1 ref 1 00000100:00100000:1.0:1789760776.001527:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786306983939 time 786342562923 state 0x0 ref 1 00000100:00100000:1.0:1789760776.001541:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493822464:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760776.001589:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26800128, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd007 00010000:00010000:1.0:1789760776.001597:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06913 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.001613:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=786s v=(0 0) 00010000:00010000:1.0:1789760776.001617:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.001623:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06913 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.001637:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06913 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.030552:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973862 is committed 00002000:00010000:3.0:1789760776.030670:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26800128 -> 26804224 00010000:00010000:3.0:1789760776.030686:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06913 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.030716:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06913 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.030724:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06913 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.030732:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760776.030737:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06913 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.030747:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06913 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.030767:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06913 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.030774:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06913 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26800128->26804223] (req 26800128->26804223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.030894:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493822464:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29350us (29644us total) trans 4294973862 rc 0/0 00000100:00100000:3.0:1789760776.030905:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6573 00000100:00100000:3.0:1789760776.030915:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786306983939 time 786342562923 state 0x2 ref 0 00000100:00100000:0.0:1789760776.034767:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760776.035059:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493822592 00000100:00100000:1.0:1789760776.035070:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493822592 00000100:00100000:1.0:1789760776.035124:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786306983939 time 786342562923 state 0x0 ref 1 00000100:00100000:1.0:1789760776.035140:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786342823339 time 786342562923 state 0x1 ref 1 00000100:00100000:1.0:1789760776.035170:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786342823339 time 786376205798 state 0x0 ref 1 00000100:00100000:1.0:1789760776.035183:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493822592:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760776.035239:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26804224, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd007 00010000:00010000:1.0:1789760776.035248:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0691a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.035268:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=786s v=(0 0) 00010000:00010000:1.0:1789760776.035404:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.035413:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0691a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.035428:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0691a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760776.061576:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26804224 -> 26808320 00010000:00010000:1.0:1789760776.061593:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0691a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.061606:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0691a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.061614:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0691a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.061622:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760776.061625:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0691a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.061634:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0691a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.061653:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0691a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.061660:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e0691a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26804224->26808319] (req 26804224->26808319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.061764:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493822592:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26579us (27002us total) trans 4294973863 rc 0/0 00000100:00100000:1.0:1789760776.061774:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6574 00000100:00100000:1.0:1789760776.061784:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786342823339 time 786376205798 state 0x2 ref 0 00000001:00080000:3.0:1789760776.062425:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973863 is committed 00000100:00100000:0.0:1789760776.066183:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760776.066507:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493822720 00000100:00100000:3.0:1789760776.066517:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493822720 00000100:00100000:3.0:1789760776.066570:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786342823339 time 786376205798 state 0x0 ref 1 00000100:00100000:3.0:1789760776.066586:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786376466214 time 786376205798 state 0x1 ref 1 00000100:00100000:3.0:1789760776.066616:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786376466214 time 786407651973 state 0x0 ref 1 00000100:00100000:3.0:1789760776.066629:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493822720:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760776.066678:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26808320, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd007 00010000:00010000:3.0:1789760776.066686:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06921 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.066702:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=786s v=(0 0) 00010000:00010000:3.0:1789760776.066706:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.066712:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06921 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.066725:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06921 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760776.097722:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26808320 -> 26812416 00010000:00010000:2.0:1789760776.097740:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06921 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.097754:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06921 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.097763:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06921 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.097771:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760776.097776:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06921 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.097785:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06921 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.097803:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06921 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.097811:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06921 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26808320->26812415] (req 26808320->26812415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.097902:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493822720:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31270us (31722us total) trans 4294973864 rc 0/0 00000100:00100000:2.0:1789760776.097912:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6575 00000100:00100000:2.0:1789760776.097922:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786376466214 time 786407651973 state 0x2 ref 0 00000001:00080000:1.0:1789760776.098706:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973864 is committed 00000100:00100000:0.0:1789760776.101814:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760776.102323:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493822848 00000100:00100000:3.0:1789760776.102340:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493822848 00000100:00100000:3.0:1789760776.102415:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786376466214 time 786407651973 state 0x0 ref 1 00000100:00100000:3.0:1789760776.102435:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786407912389 time 786407651973 state 0x1 ref 1 00000100:00100000:3.0:1789760776.102472:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786407912389 time 786443506626 state 0x0 ref 1 00000100:00100000:3.0:1789760776.102487:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493822848:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760776.102564:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26812416, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd007 00010000:00010000:3.0:1789760776.102576:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06928 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.102596:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=786s v=(0 0) 00010000:00010000:3.0:1789760776.102602:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.102610:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06928 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.102627:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06928 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.133394:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973865 is committed 00002000:00010000:0.0:1789760776.133596:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26812416 -> 26816512 00010000:00010000:0.0:1789760776.133612:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06928 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.133625:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06928 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.133634:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06928 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.133642:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760776.133647:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06928 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.133662:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06928 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.133682:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06928 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.133690:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06928 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26812416->26816511] (req 26812416->26816511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.133778:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493822848:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31288us (31967us total) trans 4294973865 rc 0/0 00000100:00100000:0.0:1789760776.133788:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6576 00000100:00100000:0.0:1789760776.133799:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786407912389 time 786443506626 state 0x2 ref 0 00000100:00100000:0.0:1789760776.138622:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760776.139456:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493822976 00000100:00100000:1.0:1789760776.139468:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493822976 00000100:00100000:1.0:1789760776.139529:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786407912389 time 786443506626 state 0x0 ref 1 00000100:00100000:1.0:1789760776.139541:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786443767042 time 786443506626 state 0x1 ref 1 00000100:00100000:1.0:1789760776.139562:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786443767042 time 786480597792 state 0x0 ref 1 00000100:00100000:1.0:1789760776.139574:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493822976:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760776.139629:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26816512, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd007 00010000:00010000:1.0:1789760776.139638:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0692f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.139653:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=786s v=(0 0) 00010000:00010000:1.0:1789760776.139658:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.139671:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0692f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.139684:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0692f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760776.174448:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26816512 -> 26820608 00010000:00010000:1.0:1789760776.174474:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0692f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.174492:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0692f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.174501:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0692f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.174510:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760776.174515:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0692f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.174525:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0692f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.174549:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0692f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.174559:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0692f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26816512->26820607] (req 26816512->26820607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.174682:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493822976:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35102us (36070us total) trans 4294973866 rc 0/0 00000100:00100000:1.0:1789760776.174695:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6577 00000100:00100000:1.0:1789760776.174707:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786443767042 time 786480597792 state 0x2 ref 0 00000001:00080000:3.0:1789760776.175665:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973866 is committed 00000100:00100000:1.0:1789760776.180466:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760776.180640:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493823104 00000100:00100000:0.0:1789760776.180656:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493823104 00000100:00100000:0.0:1789760776.180734:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786443767042 time 786480597792 state 0x0 ref 1 00000100:00100000:0.0:1789760776.180753:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786480858208 time 786480597792 state 0x1 ref 1 00000100:00100000:0.0:1789760776.180789:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786480858208 time 786521824929 state 0x0 ref 1 00000100:00100000:0.0:1789760776.180803:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493823104:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760776.180867:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26820608, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd007 00010000:00010000:0.0:1789760776.180879:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06936 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.180899:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=786s v=(0 0) 00010000:00010000:0.0:1789760776.180905:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.180912:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06936 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.180929:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06936 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.237389:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973867 is committed 00002000:00010000:1.0:1789760776.237529:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26820608 -> 26824704 00010000:00010000:1.0:1789760776.237550:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06936 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.237564:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06936 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.237574:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06936 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.237581:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760776.237587:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06936 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.237596:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06936 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.237619:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06936 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.237633:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06936 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26820608->26824703] (req 26820608->26824703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.238069:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493823104:12345-192.168.203.60@tcp:4:dd.0 Request processed in 57261us (57611us total) trans 4294973867 rc 0/0 00000100:00100000:1.0:1789760776.238082:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6578 00000100:00100000:1.0:1789760776.238095:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786480858208 time 786521824929 state 0x2 ref 0 00000100:00100000:2.0:1789760776.242380:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760776.242610:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493823232 00000100:00100000:1.0:1789760776.242624:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493823232 00000100:00100000:1.0:1789760776.242771:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786480858208 time 786521824929 state 0x0 ref 1 00000100:00100000:1.0:1789760776.242793:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786522085345 time 786521824929 state 0x1 ref 1 00000100:00100000:0.0:1789760776.242892:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786522085345 time 786583923541 state 0x0 ref 1 00000100:00100000:0.0:1789760776.242912:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493823232:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760776.243006:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26824704, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd007 00010000:00010000:0.0:1789760776.243018:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0693d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.243040:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=786s v=(0 0) 00010000:00010000:0.0:1789760776.243046:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.243052:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0693d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.243070:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0693d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.273737:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973868 is committed 00002000:00010000:1.0:1789760776.274035:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26824704 -> 26828800 00010000:00010000:1.0:1789760776.274054:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0693d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.274070:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0693d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.274080:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0693d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.274088:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760776.274094:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0693d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.274105:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0693d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.274131:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0693d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.274139:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0693d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26824704->26828799] (req 26824704->26828799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.274258:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493823232:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31346us (31887us total) trans 4294973868 rc 0/0 00000100:00100000:1.0:1789760776.274280:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6579 00000100:00100000:1.0:1789760776.274402:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786522085345 time 786583923541 state 0x2 ref 0 00000100:00100000:0.0:1789760776.280571:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760776.280760:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493823360 00000100:00100000:2.0:1789760776.280774:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493823360 00000100:00100000:2.0:1789760776.280851:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786522085345 time 786583923541 state 0x0 ref 1 00000100:00100000:2.0:1789760776.280871:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786584183957 time 786583923541 state 0x1 ref 1 00000100:00100000:2.0:1789760776.280949:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786584183957 time 786621984112 state 0x0 ref 1 00000100:00100000:2.0:1789760776.281002:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493823360:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760776.281071:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26828800, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd007 00010000:00010000:2.0:1789760776.281081:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06944 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.281102:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=786s v=(0 0) 00010000:00010000:2.0:1789760776.281108:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.281116:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06944 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.281134:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06944 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.314847:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973869 is committed 00002000:00010000:0.0:1789760776.315084:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26828800 -> 26832896 00010000:00010000:0.0:1789760776.315103:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06944 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.315118:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06944 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.315128:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06944 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.315136:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760776.315142:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06944 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.315152:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06944 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.315177:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06944 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.315185:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06944 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26828800->26832895] (req 26828800->26832895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.315315:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493823360:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34347us (34751us total) trans 4294973869 rc 0/0 00000100:00100000:0.0:1789760776.315328:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6580 00000100:00100000:0.0:1789760776.315341:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786584183957 time 786621984112 state 0x2 ref 0 00000100:00100000:0.0:1789760776.326088:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760776.326253:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493823488 00000100:00100000:3.0:1789760776.326266:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493823488 00000100:00100000:2.0:1789760776.326430:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786584183957 time 786621984112 state 0x0 ref 1 00000100:00100000:2.0:1789760776.326450:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786622244528 time 786621984112 state 0x1 ref 1 00000100:00100000:2.0:1789760776.326524:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786622244528 time 786667559305 state 0x0 ref 1 00000100:00100000:2.0:1789760776.326539:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb15c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493823488:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760776.326605:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26832896, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd007 00010000:00010000:2.0:1789760776.326627:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0694b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.326646:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=786s v=(0 0) 00010000:00010000:2.0:1789760776.326655:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.326663:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0694b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.326707:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0694b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.370453:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973870 is committed 00002000:00010000:2.0:1789760776.370587:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26832896 -> 26836992 00010000:00010000:2.0:1789760776.370606:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0694b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.370754:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0694b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.370765:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0694b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.370773:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760776.370778:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0694b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.370787:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0694b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.370812:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0694b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.370821:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0694b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26832896->26836991] (req 26832896->26836991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.371136:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb15c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493823488:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44592us (45053us total) trans 4294973870 rc 0/0 00000100:00100000:2.0:1789760776.371149:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6581 00000100:00100000:2.0:1789760776.371161:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786622244528 time 786667559305 state 0x2 ref 0 00000100:00100000:3.0:1789760776.378843:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760776.379061:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493823616 00000100:00100000:1.0:1789760776.379081:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493823616 00000100:00100000:1.0:1789760776.379182:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786622244528 time 786667559305 state 0x0 ref 1 00000100:00100000:1.0:1789760776.379204:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786667819721 time 786667559305 state 0x1 ref 1 00000100:00100000:1.0:1789760776.379289:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786667819721 time 786720323242 state 0x0 ref 1 00000100:00100000:1.0:1789760776.379305:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493823616:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760776.379375:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26836992, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd007 00010000:00010000:1.0:1789760776.379386:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06952 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.379406:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=786s v=(0 0) 00010000:00010000:1.0:1789760776.379412:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.379419:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06952 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.379437:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06952 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.413341:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973871 is committed 00002000:00010000:0.0:1789760776.413838:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26836992 -> 26841088 00010000:00010000:0.0:1789760776.413856:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06952 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.413870:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06952 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.413880:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06952 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.413888:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760776.413894:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06952 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.413904:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06952 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.413923:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06952 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.413931:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06952 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26836992->26841087] (req 26836992->26841087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.414157:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493823616:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34850us (35323us total) trans 4294973871 rc 0/0 00000100:00100000:0.0:1789760776.414439:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6582 00000100:00100000:0.0:1789760776.414451:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786667819721 time 786720323242 state 0x2 ref 0 00000100:00100000:0.0:1789760776.417089:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760776.417160:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493823744 00000100:00100000:1.0:1789760776.417169:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493823744 00000100:00100000:1.0:1789760776.417355:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786667819721 time 786720323242 state 0x0 ref 1 00000100:00100000:1.0:1789760776.417373:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786720583658 time 786720323242 state 0x1 ref 1 00000100:00100000:1.0:1789760776.417407:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786720583658 time 786758441957 state 0x0 ref 1 00000100:00100000:1.0:1789760776.417420:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493823744:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760776.417468:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26841088, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd007 00010000:00010000:1.0:1789760776.417477:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06959 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.417492:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=786s v=(0 0) 00010000:00010000:1.0:1789760776.417497:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.417504:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06959 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.417517:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06959 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760776.455437:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26841088 -> 26845184 00010000:00010000:0.0:1789760776.455458:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06959 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.455472:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06959 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.455480:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06959 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.455487:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760776.455492:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06959 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.455501:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06959 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.455521:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06959 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.455529:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06959 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26841088->26845183] (req 26841088->26845183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.455630:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493823744:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38207us (38544us total) trans 4294973872 rc 0/0 00000100:00100000:0.0:1789760776.455640:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6583 00000100:00100000:0.0:1789760776.455649:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786720583658 time 786758441957 state 0x2 ref 0 00000001:00080000:2.0:1789760776.456032:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973872 is committed 00000100:00100000:2.0:1789760776.463791:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760776.464376:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493823872 00000100:00100000:0.0:1789760776.464390:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493823872 00000100:00100000:0.0:1789760776.464451:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786720583658 time 786758441957 state 0x0 ref 1 00000100:00100000:0.0:1789760776.464466:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786758702373 time 786758441957 state 0x1 ref 1 00000100:00100000:0.0:1789760776.464553:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786758702373 time 786805587939 state 0x0 ref 1 00000100:00100000:0.0:1789760776.464567:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493823872:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760776.464617:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26845184, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd007 00010000:00010000:0.0:1789760776.464625:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06960 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.464641:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=786s v=(0 0) 00010000:00010000:0.0:1789760776.464645:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.464651:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06960 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.464666:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06960 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.492925:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973873 is committed 00002000:00010000:0.0:1789760776.493048:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26845184 -> 26849280 00010000:00010000:0.0:1789760776.493064:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06960 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.493078:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06960 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.493088:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06960 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.493095:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760776.493100:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06960 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.493109:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06960 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.493129:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06960 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.493136:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06960 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26845184->26849279] (req 26845184->26849279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.493265:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493823872:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28696us (29477us total) trans 4294973873 rc 0/0 00000100:00100000:0.0:1789760776.493289:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6584 00000100:00100000:0.0:1789760776.493298:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786758702373 time 786805587939 state 0x2 ref 0 00000100:00100000:3.0:1789760776.496788:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760776.497046:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493824000 00000100:00100000:1.0:1789760776.497056:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493824000 00000100:00100000:1.0:1789760776.497111:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786758702373 time 786805587939 state 0x0 ref 1 00000100:00100000:1.0:1789760776.497126:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786805848355 time 786805587939 state 0x1 ref 1 00000100:00100000:1.0:1789760776.497151:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786805848355 time 786838186802 state 0x0 ref 1 00000100:00100000:1.0:1789760776.497163:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493824000:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760776.497210:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26849280, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd007 00010000:00010000:1.0:1789760776.497229:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06967 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.497244:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=786s v=(0 0) 00010000:00010000:1.0:1789760776.497248:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.497254:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06967 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.497326:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06967 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.534682:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973874 is committed 00002000:00010000:3.0:1789760776.534837:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26849280 -> 26853376 00010000:00010000:3.0:1789760776.534853:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06967 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.534864:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06967 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.534877:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06967 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.534884:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760776.534888:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06967 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.534895:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06967 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.534914:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06967 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.534922:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06967 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26849280->26853375] (req 26849280->26853375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.535198:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493824000:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38032us (38415us total) trans 4294973874 rc 0/0 00000100:00100000:3.0:1789760776.535208:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6585 00000100:00100000:3.0:1789760776.535230:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786805848355 time 786838186802 state 0x2 ref 0 00000100:00100000:1.0:1789760776.538152:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760776.538254:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493824128 00000100:00100000:0.0:1789760776.538263:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493824128 00000100:00100000:0.0:1789760776.538377:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786805848355 time 786838186802 state 0x0 ref 1 00000100:00100000:0.0:1789760776.538393:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786838447218 time 786838186802 state 0x1 ref 1 00000100:00100000:0.0:1789760776.538470:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786838447218 time 786879504665 state 0x0 ref 1 00000100:00100000:0.0:1789760776.538483:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493824128:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760776.538529:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26853376, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd007 00010000:00010000:0.0:1789760776.538538:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0696e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.538553:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=786s v=(0 0) 00010000:00010000:0.0:1789760776.538557:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.538563:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0696e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.538576:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0696e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.567770:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973875 is committed 00002000:00010000:2.0:1789760776.567882:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26853376 -> 26857472 00010000:00010000:2.0:1789760776.567898:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0696e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.567910:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0696e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.567919:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0696e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.567926:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760776.567930:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0696e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.567940:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0696e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.567955:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0696e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.567986:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e0696e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26853376->26857471] (req 26853376->26857471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.568077:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493824128:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29591us (29929us total) trans 4294973875 rc 0/0 00000100:00100000:2.0:1789760776.568095:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6586 00000100:00100000:2.0:1789760776.568108:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786838447218 time 786879504665 state 0x2 ref 0 00000100:00100000:0.0:1789760776.572320:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760776.572390:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493824256 00000100:00100000:1.0:1789760776.572401:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493824256 00000100:00100000:1.0:1789760776.572454:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786838447218 time 786879504665 state 0x0 ref 1 00000100:00100000:1.0:1789760776.572470:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786879765081 time 786879504665 state 0x1 ref 1 00000100:00100000:1.0:1789760776.572635:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786879765081 time 786913668432 state 0x0 ref 1 00000100:00100000:1.0:1789760776.572648:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493824256:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760776.572753:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26857472, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xd007 00010000:00010000:1.0:1789760776.572764:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06975 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.572779:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=786s v=(0 0) 00010000:00010000:1.0:1789760776.572784:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.572789:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06975 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.572801:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06975 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.611839:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973876 is committed 00002000:00010000:0.0:1789760776.611949:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26857472 -> 26861568 00002000:00010000:0.0:1789760776.611954:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760775 -> 1789760776 00002000:00010000:0.0:1789760776.611956:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760775 -> 1789760776 00010000:00010000:0.0:1789760776.611989:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06975 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.612002:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06975 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.612012:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06975 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.612019:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760776.612024:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06975 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.612033:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06975 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.612053:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06975 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.612061:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06975 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26857472->26861567] (req 26857472->26861567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.612160:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493824256:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39509us (39844us total) trans 4294973876 rc 0/0 00000100:00100000:0.0:1789760776.612175:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6587 00000100:00100000:0.0:1789760776.612186:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786879765081 time 786913668432 state 0x2 ref 0 00000100:00100000:3.0:1789760776.623055:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760776.623121:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493824384 00000100:00100000:2.0:1789760776.623130:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493824384 00000100:00100000:2.0:1789760776.623186:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786879765081 time 786913668432 state 0x0 ref 1 00000100:00100000:2.0:1789760776.623200:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786913928848 time 786913668432 state 0x1 ref 1 00000100:00100000:2.0:1789760776.623234:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786913928848 time 786964269127 state 0x0 ref 1 00000100:00100000:2.0:1789760776.623247:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493824384:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760776.623295:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26861568, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd007 00010000:00010000:2.0:1789760776.623368:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0697c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.623385:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=786s v=(0 0) 00010000:00010000:2.0:1789760776.623390:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.623395:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0697c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.623409:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0697c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.656565:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973877 is committed 00002000:00010000:3.0:1789760776.656743:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26861568 -> 26865664 00010000:00010000:3.0:1789760776.656759:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0697c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.656776:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0697c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.656785:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0697c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.656793:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760776.656797:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0697c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.656806:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0697c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.656826:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0697c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.656834:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0697c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26861568->26865663] (req 26861568->26865663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.656937:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493824384:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33688us (33887us total) trans 4294973877 rc 0/0 00000100:00100000:3.0:1789760776.656947:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6588 00000100:00100000:3.0:1789760776.656999:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786913928848 time 786964269127 state 0x2 ref 0 00000100:00100000:2.0:1789760776.662616:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760776.663017:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493824512 00000100:00100000:3.0:1789760776.663028:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493824512 00000100:00100000:3.0:1789760776.663085:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786913928848 time 786964269127 state 0x0 ref 1 00000100:00100000:3.0:1789760776.663107:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786964529543 time 786964269127 state 0x1 ref 1 00000100:00100000:3.0:1789760776.663127:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786964529543 time 787004163721 state 0x0 ref 1 00000100:00100000:3.0:1789760776.663140:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493824512:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760776.663185:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26865664, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd007 00010000:00010000:3.0:1789760776.663194:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06983 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.663210:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=787s v=(0 0) 00010000:00010000:3.0:1789760776.663224:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.663230:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06983 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.663246:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06983 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.695065:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973878 is committed 00002000:00010000:3.0:1789760776.695181:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26865664 -> 26869760 00010000:00010000:3.0:1789760776.695196:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06983 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.695548:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06983 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.695561:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06983 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.695566:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760776.695569:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06983 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.695575:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06983 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.695592:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06983 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.695596:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06983 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26865664->26869759] (req 26865664->26869759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.695752:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493824512:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32611us (33142us total) trans 4294973878 rc 0/0 00000100:00100000:3.0:1789760776.695761:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6589 00000100:00100000:3.0:1789760776.695768:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786964529543 time 787004163721 state 0x2 ref 0 00000100:00100000:0.0:1789760776.702191:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760776.703011:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493824640 00000100:00100000:2.0:1789760776.703020:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493824640 00000100:00100000:2.0:1789760776.703075:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 786964529543 time 787004163721 state 0x0 ref 1 00000100:00100000:2.0:1789760776.703090:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787004424137 time 787004163721 state 0x1 ref 1 00000100:00100000:2.0:1789760776.703122:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787004424137 time 787044157186 state 0x0 ref 1 00000100:00100000:2.0:1789760776.703134:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493824640:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760776.703180:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26869760, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd007 00010000:00010000:2.0:1789760776.703188:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0698a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.703204:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=787s v=(0 0) 00010000:00010000:2.0:1789760776.703208:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.703224:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0698a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.703609:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0698a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.741191:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973879 is committed 00002000:00010000:3.0:1789760776.741487:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26869760 -> 26873856 00010000:00010000:3.0:1789760776.741502:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0698a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.741519:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0698a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.741528:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0698a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.741535:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760776.741540:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0698a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.741549:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0698a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.741568:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0698a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.741574:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0698a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26869760->26873855] (req 26869760->26873855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.741663:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493824640:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38525us (39482us total) trans 4294973879 rc 0/0 00000100:00100000:3.0:1789760776.741673:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6590 00000100:00100000:3.0:1789760776.741681:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787004424137 time 787044157186 state 0x2 ref 0 00000100:00100000:0.0:1789760776.746773:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760776.746856:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493824768 00000100:00100000:1.0:1789760776.746866:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493824768 00000100:00100000:1.0:1789760776.746922:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787004424137 time 787044157186 state 0x0 ref 1 00000100:00100000:1.0:1789760776.746945:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787044417602 time 787044157186 state 0x1 ref 1 00000100:00100000:1.0:1789760776.747006:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787044417602 time 787088035065 state 0x0 ref 1 00000100:00100000:1.0:1789760776.747020:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493824768:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760776.747069:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26873856, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd007 00010000:00010000:1.0:1789760776.747078:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06991 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.747097:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=787s v=(0 0) 00010000:00010000:1.0:1789760776.747103:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.747109:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06991 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.747124:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06991 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.763925:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760776.810649:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26873856 -> 26877952 00000001:00080000:1.0:1789760776.810665:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973880 is committed 00010000:00010000:2.0:1789760776.810669:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06991 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.810680:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06991 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.810695:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06991 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.810703:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760776.810708:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06991 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.810716:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06991 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.810734:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06991 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.810743:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06991 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26873856->26877951] (req 26873856->26877951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.810858:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493824768:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63837us (64092us total) trans 4294973880 rc 0/0 00000100:00100000:2.0:1789760776.810868:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6591 00000100:00100000:2.0:1789760776.810877:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787044417602 time 787088035065 state 0x2 ref 0 00000100:00100000:1.0:1789760776.817135:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760776.818361:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493824896 00000100:00100000:3.0:1789760776.818370:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493824896 00000100:00100000:3.0:1789760776.818424:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787044417602 time 787088035065 state 0x0 ref 1 00000100:00100000:3.0:1789760776.818439:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787088295481 time 787088035065 state 0x1 ref 1 00000100:00100000:3.0:1789760776.818473:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787088295481 time 787159509019 state 0x0 ref 1 00000100:00100000:3.0:1789760776.818485:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493824896:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760776.818532:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26877952, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd007 00010000:00010000:3.0:1789760776.818541:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06998 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.818556:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=787s v=(0 0) 00010000:00010000:3.0:1789760776.818560:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.818566:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06998 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.818580:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06998 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.846117:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973881 is committed 00002000:00010000:2.0:1789760776.846303:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26877952 -> 26882048 00010000:00010000:2.0:1789760776.846318:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06998 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.846337:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06998 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.846345:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06998 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.846353:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760776.846358:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06998 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.846371:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06998 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.846389:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06998 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.846397:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06998 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26877952->26882047] (req 26877952->26882047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.846486:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493824896:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27997us (29358us total) trans 4294973881 rc 0/0 00000100:00100000:2.0:1789760776.846496:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6592 00000100:00100000:2.0:1789760776.846506:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787088295481 time 787159509019 state 0x2 ref 0 00000100:00100000:0.0:1789760776.850331:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760776.850441:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493825024 00000100:00100000:3.0:1789760776.850450:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493825024 00000100:00100000:3.0:1789760776.850502:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787088295481 time 787159509019 state 0x0 ref 1 00000100:00100000:3.0:1789760776.850516:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787159769435 time 787159509019 state 0x1 ref 1 00000100:00100000:3.0:1789760776.850544:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787159769435 time 787191580103 state 0x0 ref 1 00000100:00100000:3.0:1789760776.850555:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493825024:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760776.850603:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26882048, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd007 00010000:00010000:3.0:1789760776.850612:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0699f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.850626:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=787s v=(0 0) 00010000:00010000:3.0:1789760776.850631:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.850637:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0699f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.850650:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0699f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.880068:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973882 is committed 00002000:00010000:2.0:1789760776.880240:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26882048 -> 26886144 00010000:00010000:2.0:1789760776.880256:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0699f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.880277:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0699f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.880286:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0699f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.880293:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760776.880299:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0699f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.880308:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0699f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.880326:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0699f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.880334:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0699f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26882048->26886143] (req 26882048->26886143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.880423:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493825024:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29864us (30095us total) trans 4294973882 rc 0/0 00000100:00100000:2.0:1789760776.880434:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6593 00000100:00100000:2.0:1789760776.880444:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787159769435 time 787191580103 state 0x2 ref 0 00000100:00100000:0.0:1789760776.894042:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760776.894150:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493825152 00000100:00100000:2.0:1789760776.894161:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493825152 00000100:00100000:2.0:1789760776.894221:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787159769435 time 787191580103 state 0x0 ref 1 00000100:00100000:2.0:1789760776.894237:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787191840519 time 787191580103 state 0x1 ref 1 00000100:00100000:2.0:1789760776.894314:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787191840519 time 787235348300 state 0x0 ref 1 00000100:00100000:2.0:1789760776.894328:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493825152:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760776.894378:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26886144, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd007 00010000:00010000:2.0:1789760776.894386:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e069a6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.894402:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=787s v=(0 0) 00010000:00010000:2.0:1789760776.894406:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.894412:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e069a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.894424:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e069a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.925337:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973883 is committed 00002000:00010000:2.0:1789760776.925439:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26886144 -> 26890240 00010000:00010000:2.0:1789760776.925454:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e069a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.925467:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e069a6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.925477:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e069a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.925485:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760776.925490:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e069a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.925499:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e069a6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.925518:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e069a6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.925526:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e069a6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26886144->26890239] (req 26886144->26890239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.925622:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493825152:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31292us (31585us total) trans 4294973883 rc 0/0 00000100:00100000:2.0:1789760776.925633:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6594 00000100:00100000:2.0:1789760776.925643:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787191840519 time 787235348300 state 0x2 ref 0 00000100:00100000:1.0:1789760776.933116:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760776.933592:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493825280 00000100:00100000:2.0:1789760776.933602:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493825280 00000100:00100000:2.0:1789760776.933656:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787191840519 time 787235348300 state 0x0 ref 1 00000100:00100000:2.0:1789760776.933672:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787235608716 time 787235348300 state 0x1 ref 1 00000100:00100000:2.0:1789760776.933801:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787235608716 time 787274836023 state 0x0 ref 1 00000100:00100000:2.0:1789760776.933814:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493825280:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760776.933861:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26890240, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd007 00010000:00010000:2.0:1789760776.933869:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e069ad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.933884:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=787s v=(0 0) 00010000:00010000:2.0:1789760776.933888:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.933894:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e069ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.933907:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e069ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.965664:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973884 is committed 00002000:00010000:2.0:1789760776.965814:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26890240 -> 26894336 00010000:00010000:2.0:1789760776.965830:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e069ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.965843:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e069ad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.965852:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e069ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.965859:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760776.965864:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e069ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.965872:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e069ad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.965891:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e069ad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.965898:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e069ad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26890240->26894335] (req 26890240->26894335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.966124:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493825280:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32307us (33013us total) trans 4294973884 rc 0/0 00000100:00100000:2.0:1789760776.966135:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6595 00000100:00100000:2.0:1789760776.966144:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787235608716 time 787274836023 state 0x2 ref 0 00000100:00100000:1.0:1789760776.973860:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760776.973954:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493825408 00000100:00100000:3.0:1789760776.973996:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493825408 00000100:00100000:3.0:1789760776.974047:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787235608716 time 787274836023 state 0x0 ref 1 00000100:00100000:3.0:1789760776.974062:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787275096439 time 787274836023 state 0x1 ref 1 00000100:00100000:3.0:1789760776.974094:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787275096439 time 787315129804 state 0x0 ref 1 00000100:00100000:3.0:1789760776.974106:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493825408:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760776.974153:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26894336, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd007 00010000:00010000:3.0:1789760776.974162:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e069b4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.974182:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=787s v=(0 0) 00010000:00010000:3.0:1789760776.974187:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.974193:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e069b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.974206:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e069b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.031486:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973885 is committed 00002000:00010000:1.0:1789760777.031611:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26894336 -> 26898432 00010000:00010000:1.0:1789760777.031625:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e069b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.031638:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e069b4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.031646:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e069b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.031653:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760777.031657:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e069b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.031666:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e069b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.031826:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e069b4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.031836:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e069b4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26894336->26898431] (req 26894336->26898431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.032005:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493825408:12345-192.168.203.60@tcp:4:dd.0 Request processed in 57897us (58150us total) trans 4294973885 rc 0/0 00000100:00100000:1.0:1789760777.032017:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6596 00000100:00100000:1.0:1789760777.032027:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787275096439 time 787315129804 state 0x2 ref 0 00000100:00100000:3.0:1789760777.035942:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760777.036154:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493825536 00000100:00100000:2.0:1789760777.036163:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493825536 00000100:00080000:1.0:1789760777.036206:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760777.036240:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787275096439 time 787315129804 state 0x0 ref 1 00000100:00100000:2.0:1789760777.036265:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787315390220 time 787315129804 state 0x1 ref 1 00000100:00100000:2.0:1789760777.036294:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787315390220 time 787377329625 state 0x0 ref 1 00000100:00100000:2.0:1789760777.036307:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493825536:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760777.036358:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26898432, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd007 00010000:00010000:2.0:1789760777.036367:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e069bb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.036381:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=787s v=(0 0) 00010000:00010000:2.0:1789760777.036385:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.036391:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e069bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.036405:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e069bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760777.066201:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26898432 -> 26902528 00010000:00010000:2.0:1789760777.066231:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e069bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.066245:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e069bb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.066254:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e069bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.066262:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760777.066275:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e069bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.066285:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e069bb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.066304:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e069bb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.066312:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e069bb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26898432->26902527] (req 26898432->26902527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.066466:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493825536:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30156us (30526us total) trans 4294973886 rc 0/0 00000100:00100000:2.0:1789760777.066476:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6597 00000100:00100000:2.0:1789760777.066486:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787315390220 time 787377329625 state 0x2 ref 0 00000001:00080000:2.0:1789760777.066521:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973886 is committed 00000100:00100000:2.0:1789760777.074812:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760777.075476:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493825664 00000100:00100000:3.0:1789760777.075489:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493825664 00000100:00100000:2.0:1789760777.078073:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787315390220 time 787377329625 state 0x0 ref 1 00000100:00100000:2.0:1789760777.078093:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787377590041 time 787377329625 state 0x1 ref 1 00000100:00100000:2.0:1789760777.078191:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787377590041 time 787419225714 state 0x0 ref 1 00000100:00100000:2.0:1789760777.078206:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493825664:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760777.078265:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26902528, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd007 00010000:00010000:2.0:1789760777.078273:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e069c2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.078289:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=787s v=(0 0) 00010000:00010000:2.0:1789760777.078294:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.078300:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e069c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.078314:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e069c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.117892:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973887 is committed 00002000:00010000:3.0:1789760777.118156:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26902528 -> 26906624 00010000:00010000:3.0:1789760777.118170:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e069c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.118182:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e069c2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.118191:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e069c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.118198:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760777.118202:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e069c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.118226:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e069c2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.118247:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e069c2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.118256:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e069c2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26902528->26906623] (req 26902528->26906623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.118452:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493825664:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40245us (43645us total) trans 4294973887 rc 0/0 00000100:00100000:3.0:1789760777.118462:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6598 00000100:00100000:3.0:1789760777.118472:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787377590041 time 787419225714 state 0x2 ref 0 00000100:00100000:2.0:1789760777.125328:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760777.125416:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493825792 00000100:00100000:1.0:1789760777.125425:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493825792 00000100:00100000:1.0:1789760777.125479:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787377590041 time 787419225714 state 0x0 ref 1 00000100:00100000:1.0:1789760777.125494:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787419486130 time 787419225714 state 0x1 ref 1 00000100:00100000:1.0:1789760777.125525:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787419486130 time 787466560384 state 0x0 ref 1 00000100:00100000:1.0:1789760777.125537:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493825792:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760777.125583:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26906624, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd007 00010000:00010000:1.0:1789760777.125592:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e069c9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.125613:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=787s v=(0 0) 00010000:00010000:1.0:1789760777.125618:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.125624:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e069c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.125637:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e069c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.148529:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973888 is committed 00002000:00010000:0.0:1789760777.148805:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26906624 -> 26910720 00010000:00010000:0.0:1789760777.148820:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e069c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.148833:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e069c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.148842:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e069c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.148850:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760777.148855:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e069c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.148865:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e069c9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.148884:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e069c9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.148893:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e069c9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26906624->26910719] (req 26906624->26910719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.149458:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493825792:12345-192.168.203.60@tcp:4:dd.0 Request processed in 23915us (24132us total) trans 4294973888 rc 0/0 00000100:00100000:0.0:1789760777.149469:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6599 00000100:00100000:0.0:1789760777.149479:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787419486130 time 787466560384 state 0x2 ref 0 00000100:00100000:0.0:1789760777.152833:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760777.153219:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493825920 00000100:00100000:1.0:1789760777.153229:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493825920 00000100:00100000:1.0:1789760777.153290:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787419486130 time 787466560384 state 0x0 ref 1 00000100:00100000:1.0:1789760777.153308:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787466820800 time 787466560384 state 0x1 ref 1 00000100:00100000:1.0:1789760777.153339:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787466820800 time 787494374641 state 0x0 ref 1 00000100:00100000:1.0:1789760777.153352:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493825920:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760777.153399:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26910720, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd007 00010000:00010000:1.0:1789760777.153408:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e069d0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.153423:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=787s v=(0 0) 00010000:00010000:1.0:1789760777.153428:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.153435:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e069d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.153449:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e069d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.183900:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973889 is committed 00002000:00010000:0.0:1789760777.184046:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26910720 -> 26914816 00010000:00010000:0.0:1789760777.184061:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e069d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.184074:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e069d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.184083:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e069d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.184092:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760777.184097:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e069d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.184106:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e069d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.184132:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e069d0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.184140:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e069d0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26910720->26914815] (req 26910720->26914815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.184271:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493825920:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30910us (31436us total) trans 4294973889 rc 0/0 00000100:00100000:0.0:1789760777.184380:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6600 00000100:00100000:0.0:1789760777.184388:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787466820800 time 787494374641 state 0x2 ref 0 00000100:00100000:1.0:1789760777.187207:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760777.187412:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493826048 00000100:00100000:2.0:1789760777.187421:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493826048 00000100:00100000:2.0:1789760777.187476:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787466820800 time 787494374641 state 0x0 ref 1 00000100:00100000:2.0:1789760777.187491:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787494635057 time 787494374641 state 0x1 ref 1 00000100:00100000:2.0:1789760777.187549:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787494635057 time 787528584830 state 0x0 ref 1 00000100:00100000:2.0:1789760777.187562:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493826048:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760777.187619:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26914816, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd007 00010000:00010000:2.0:1789760777.187627:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e069d7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.187643:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=787s v=(0 0) 00010000:00010000:2.0:1789760777.187647:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.187653:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e069d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.187666:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e069d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760777.232744:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26914816 -> 26918912 00010000:00010000:3.0:1789760777.232765:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e069d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.232778:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e069d7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.232787:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e069d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.232800:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760777.232805:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e069d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.232813:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e069d7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.232832:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e069d7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.232840:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e069d7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26914816->26918911] (req 26914816->26918911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.232998:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493826048:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45399us (45760us total) trans 4294973890 rc 0/0 00000100:00100000:3.0:1789760777.233009:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6601 00000100:00100000:3.0:1789760777.233019:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787494635057 time 787528584830 state 0x2 ref 0 00000001:00080000:0.0:1789760777.233132:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973890 is committed 00000100:00100000:2.0:1789760777.236830:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760777.236924:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493826176 00000100:00100000:3.0:1789760777.236933:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493826176 00000100:00100000:3.0:1789760777.237242:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787494635057 time 787528584830 state 0x0 ref 1 00000100:00100000:3.0:1789760777.237370:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787528845246 time 787528584830 state 0x1 ref 1 00000100:00100000:2.0:1789760777.237487:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787528845246 time 787578520336 state 0x0 ref 1 00000100:00100000:2.0:1789760777.237503:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8794700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493826176:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760777.237549:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26918912, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd007 00010000:00010000:2.0:1789760777.237557:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e069de lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.237572:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=787s v=(0 0) 00010000:00010000:2.0:1789760777.237576:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.237582:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e069de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.237595:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e069de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.269901:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973891 is committed 00002000:00010000:2.0:1789760777.270015:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26918912 -> 26923008 00010000:00010000:2.0:1789760777.270031:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e069de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.270046:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e069de lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.270055:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e069de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.270063:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760777.270069:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e069de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.270078:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e069de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.270097:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e069de lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.270105:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e069de lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26918912->26923007] (req 26918912->26923007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.270196:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8794700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493826176:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32692us (33370us total) trans 4294973891 rc 0/0 00000100:00100000:2.0:1789760777.270206:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6602 00000100:00100000:2.0:1789760777.270227:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787528845246 time 787578520336 state 0x2 ref 0 00000100:00100000:1.0:1789760777.275757:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760777.276001:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493826304 00000100:00100000:0.0:1789760777.276010:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493826304 00000100:00100000:0.0:1789760777.276067:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787528845246 time 787578520336 state 0x0 ref 1 00000100:00100000:0.0:1789760777.276082:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787578780752 time 787578520336 state 0x1 ref 1 00000100:00100000:3.0:1789760777.276271:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787578780752 time 787617303634 state 0x0 ref 1 00000100:00100000:3.0:1789760777.276289:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493826304:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760777.276337:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26923008, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd007 00010000:00010000:3.0:1789760777.276346:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e069e5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.276361:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=787s v=(0 0) 00010000:00010000:3.0:1789760777.276366:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.276373:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e069e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.276385:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e069e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760777.318117:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26923008 -> 26927104 00010000:00010000:3.0:1789760777.318137:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e069e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.318150:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e069e5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.318160:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e069e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.318178:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760777.318183:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e069e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.318193:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e069e5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.318221:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e069e5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.318229:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e069e5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26923008->26927103] (req 26923008->26927103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.318329:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493826304:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42037us (42575us total) trans 4294973892 rc 0/0 00000100:00100000:3.0:1789760777.318340:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6603 00000100:00100000:3.0:1789760777.318359:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787578780752 time 787617303634 state 0x2 ref 0 00000001:00080000:2.0:1789760777.321586:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973892 is committed 00000100:00100000:1.0:1789760777.322479:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760777.322571:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493826432 00000100:00100000:0.0:1789760777.322580:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493826432 00000100:00100000:0.0:1789760777.322630:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787578780752 time 787617303634 state 0x0 ref 1 00000100:00100000:0.0:1789760777.322644:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787617564050 time 787617303634 state 0x1 ref 1 00000100:00100000:0.0:1789760777.322669:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787617564050 time 787663704992 state 0x0 ref 1 00000100:00100000:0.0:1789760777.322687:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493826432:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760777.322734:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26927104, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd007 00010000:00010000:0.0:1789760777.322744:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e069ec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.322759:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=787s v=(0 0) 00010000:00010000:0.0:1789760777.322764:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.322770:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e069ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.322784:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e069ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760777.355621:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26927104 -> 26931200 00010000:00010000:2.0:1789760777.355639:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e069ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.355651:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e069ec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.355664:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e069ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.355671:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760777.355676:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e069ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.355685:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e069ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.355704:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e069ec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.355712:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e069ec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26927104->26931199] (req 26927104->26931199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.355802:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493826432:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33113us (33323us total) trans 4294973893 rc 0/0 00000100:00100000:2.0:1789760777.355812:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6604 00000100:00100000:2.0:1789760777.355821:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787617564050 time 787663704992 state 0x2 ref 0 00000001:00080000:3.0:1789760777.355890:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973893 is committed 00000100:00100000:0.0:1789760777.359035:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760777.359160:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493826560 00000100:00100000:1.0:1789760777.359178:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493826560 00000100:00100000:1.0:1789760777.359246:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787617564050 time 787663704992 state 0x0 ref 1 00000100:00100000:1.0:1789760777.359261:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787663965408 time 787663704992 state 0x1 ref 1 00000100:00100000:1.0:1789760777.359299:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787663965408 time 787700334769 state 0x0 ref 1 00000100:00100000:1.0:1789760777.359311:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493826560:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760777.359359:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26931200, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd007 00010000:00010000:1.0:1789760777.359369:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e069f3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.359383:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=787s v=(0 0) 00010000:00010000:1.0:1789760777.359388:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.359394:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e069f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.359406:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e069f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760777.393487:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26931200 -> 26935296 00010000:00010000:2.0:1789760777.393513:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e069f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.393523:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973894 is committed 00010000:00010000:2.0:1789760777.393531:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e069f3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.393539:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e069f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.393558:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760777.393563:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e069f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.393573:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e069f3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.393592:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e069f3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.393600:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e069f3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26931200->26935295] (req 26931200->26935295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.393817:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493826560:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34503us (34786us total) trans 4294973894 rc 0/0 00000100:00100000:2.0:1789760777.393828:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6605 00000100:00100000:2.0:1789760777.393837:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787663965408 time 787700334769 state 0x2 ref 0 00000100:00100000:0.0:1789760777.397531:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760777.397645:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493826688 00000100:00100000:2.0:1789760777.397655:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493826688 00000100:00100000:2.0:1789760777.397766:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787663965408 time 787700334769 state 0x0 ref 1 00000100:00100000:2.0:1789760777.397781:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787700595185 time 787700334769 state 0x1 ref 1 00000100:00100000:2.0:1789760777.397858:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787700595185 time 787738892465 state 0x0 ref 1 00000100:00100000:2.0:1789760777.397872:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493826688:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760777.397919:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26935296, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd007 00010000:00010000:2.0:1789760777.397928:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e069fa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.397943:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=787s v=(0 0) 00010000:00010000:2.0:1789760777.397947:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.397953:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e069fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.398121:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e069fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.435999:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973895 is committed 00002000:00010000:2.0:1789760777.436108:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26935296 -> 26939392 00010000:00010000:2.0:1789760777.436122:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e069fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.436134:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e069fa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.436148:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e069fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.436155:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760777.436159:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e069fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.436169:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e069fa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.436189:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e069fa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.436197:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e069fa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26935296->26939391] (req 26935296->26939391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.436305:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493826688:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38432us (38780us total) trans 4294973895 rc 0/0 00000100:00100000:2.0:1789760777.436315:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6606 00000100:00100000:2.0:1789760777.436324:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787700595185 time 787738892465 state 0x2 ref 0 00000100:00100000:1.0:1789760777.440880:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760777.441145:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493826816 00000100:00100000:3.0:1789760777.441154:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493826816 00000100:00100000:3.0:1789760777.441206:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787700595185 time 787738892465 state 0x0 ref 1 00000100:00100000:3.0:1789760777.441231:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787739152881 time 787738892465 state 0x1 ref 1 00000100:00100000:3.0:1789760777.441260:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787739152881 time 787782295389 state 0x0 ref 1 00000100:00100000:3.0:1789760777.441372:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493826816:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760777.441417:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26939392, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd007 00010000:00010000:3.0:1789760777.441426:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06a01 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.441442:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=787s v=(0 0) 00010000:00010000:3.0:1789760777.441447:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.441453:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06a01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.441467:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06a01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760777.467845:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26939392 -> 26943488 00010000:00010000:2.0:1789760777.467864:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06a01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.467882:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06a01 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.467891:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06a01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.467899:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760777.467904:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06a01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.467913:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06a01 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.467931:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06a01 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.467940:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06a01 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26939392->26943487] (req 26939392->26943487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.468070:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493826816:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26698us (27195us total) trans 4294973896 rc 0/0 00000100:00100000:2.0:1789760777.468081:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6607 00000100:00100000:2.0:1789760777.468091:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787739152881 time 787782295389 state 0x2 ref 0 00000001:00080000:0.0:1789760777.469244:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973896 is committed 00000100:00100000:0.0:1789760777.474065:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760777.474219:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493826944 00000100:00100000:2.0:1789760777.474228:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493826944 00000100:00100000:2.0:1789760777.474287:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787739152881 time 787782295389 state 0x0 ref 1 00000100:00100000:2.0:1789760777.474302:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787782555805 time 787782295389 state 0x1 ref 1 00000100:00100000:2.0:1789760777.474332:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787782555805 time 787815368122 state 0x0 ref 1 00000100:00100000:2.0:1789760777.474344:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493826944:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760777.474393:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26943488, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd007 00010000:00010000:2.0:1789760777.474402:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06a08 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.474416:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=787s v=(0 0) 00010000:00010000:2.0:1789760777.474421:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.474427:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06a08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.474443:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06a08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.512505:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973897 is committed 00002000:00010000:2.0:1789760777.512625:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26943488 -> 26947584 00010000:00010000:2.0:1789760777.512639:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06a08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.512651:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06a08 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.512666:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06a08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.512674:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760777.512678:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06a08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.512752:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06a08 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.512773:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06a08 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.512781:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06a08 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26943488->26947583] (req 26943488->26947583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.512871:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493826944:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38525us (38811us total) trans 4294973897 rc 0/0 00000100:00100000:2.0:1789760777.512882:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6608 00000100:00100000:2.0:1789760777.512891:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787782555805 time 787815368122 state 0x2 ref 0 00000100:00100000:2.0:1789760777.516237:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760777.516416:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493827072 00000100:00100000:1.0:1789760777.516426:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493827072 00000100:00100000:1.0:1789760777.516479:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787782555805 time 787815368122 state 0x0 ref 1 00000100:00100000:1.0:1789760777.516494:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787815628538 time 787815368122 state 0x1 ref 1 00000100:00100000:1.0:1789760777.516562:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787815628538 time 787857596222 state 0x0 ref 1 00000100:00100000:1.0:1789760777.516575:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8796a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493827072:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760777.516626:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26947584, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd007 00010000:00010000:1.0:1789760777.516635:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06a0f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.516650:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=787s v=(0 0) 00010000:00010000:1.0:1789760777.516654:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.516660:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06a0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.516673:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06a0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.544864:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973898 is committed 00002000:00010000:2.0:1789760777.544994:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26947584 -> 26951680 00010000:00010000:2.0:1789760777.545012:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06a0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.545030:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06a0f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.545039:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06a0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.545046:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760777.545051:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06a0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.545059:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06a0f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.545078:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06a0f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.545085:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06a0f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26947584->26951679] (req 26947584->26951679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.545178:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8796a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493827072:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28601us (28947us total) trans 4294973898 rc 0/0 00000100:00100000:2.0:1789760777.545188:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6609 00000100:00100000:2.0:1789760777.545197:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787815628538 time 787857596222 state 0x2 ref 0 00000100:00100000:3.0:1789760777.550632:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760777.551181:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493827200 00000100:00100000:0.0:1789760777.551191:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493827200 00000100:00100000:0.0:1789760777.551262:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787815628538 time 787857596222 state 0x0 ref 1 00000100:00100000:0.0:1789760777.551278:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787857856638 time 787857596222 state 0x1 ref 1 00000100:00100000:0.0:1789760777.551309:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787857856638 time 787892345269 state 0x0 ref 1 00000100:00100000:0.0:1789760777.551323:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493827200:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760777.551377:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26951680, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd007 00010000:00010000:0.0:1789760777.551387:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06a16 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.551402:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=787s v=(0 0) 00010000:00010000:0.0:1789760777.551407:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.551413:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06a16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.551427:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06a16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.577698:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973899 is committed 00002000:00010000:0.0:1789760777.577806:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26951680 -> 26955776 00010000:00010000:0.0:1789760777.577822:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06a16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.577835:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06a16 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.577843:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06a16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.577850:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760777.577854:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06a16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.577862:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06a16 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.577882:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06a16 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.577901:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06a16 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26951680->26955775] (req 26951680->26955775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.578019:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493827200:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26694us (27389us total) trans 4294973899 rc 0/0 00000100:00100000:0.0:1789760777.578031:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6610 00000100:00100000:0.0:1789760777.578045:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787857856638 time 787892345269 state 0x2 ref 0 00000100:00100000:3.0:1789760777.583406:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760777.583646:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493827328 00000100:00100000:2.0:1789760777.583656:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493827328 00000100:00100000:2.0:1789760777.584093:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787857856638 time 787892345269 state 0x0 ref 1 00000100:00100000:2.0:1789760777.584108:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787892605685 time 787892345269 state 0x1 ref 1 00000100:00100000:2.0:1789760777.584142:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787892605685 time 787925178013 state 0x0 ref 1 00000100:00100000:2.0:1789760777.584156:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493827328:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760777.584204:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26955776, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xd007 00010000:00010000:2.0:1789760777.584227:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06a1d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.584241:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=787s v=(0 0) 00010000:00010000:2.0:1789760777.584246:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.584252:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06a1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.584275:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06a1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760777.635122:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26955776 -> 26959872 00000001:00080000:2.0:1789760777.635127:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973900 is committed 00002000:00010000:0.0:1789760777.635130:0:13038:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760776 -> 1789760777 00002000:00010000:0.0:1789760777.635132:0:13038:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760776 -> 1789760777 00010000:00010000:0.0:1789760777.635145:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06a1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.635158:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06a1d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.635166:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06a1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.635179:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760777.635184:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06a1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.635194:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06a1d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.635226:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06a1d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.635233:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06a1d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26955776->26959871] (req 26955776->26959871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.635400:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493827328:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51245us (51997us total) trans 4294973900 rc 0/0 00000100:00100000:0.0:1789760777.635408:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6611 00000100:00100000:0.0:1789760777.635416:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787892605685 time 787925178013 state 0x2 ref 0 00000100:00100000:1.0:1789760777.639045:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760777.639197:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493827456 00000100:00100000:2.0:1789760777.639215:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493827456 00000100:00100000:2.0:1789760777.639396:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787892605685 time 787925178013 state 0x0 ref 1 00000100:00100000:2.0:1789760777.639412:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787925438429 time 787925178013 state 0x1 ref 1 00000100:00100000:2.0:1789760777.639477:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787925438429 time 787980512444 state 0x0 ref 1 00000100:00100000:2.0:1789760777.639491:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493827456:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760777.639541:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26959872, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd007 00010000:00010000:2.0:1789760777.639549:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06a24 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.639565:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=787s v=(0 0) 00010000:00010000:2.0:1789760777.639569:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.639575:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06a24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.639588:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06a24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.668325:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973901 is committed 00002000:00010000:0.0:1789760777.668427:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26959872 -> 26963968 00010000:00010000:0.0:1789760777.668443:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06a24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.668456:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06a24 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.668472:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06a24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.668480:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760777.668486:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06a24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.668495:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06a24 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.668516:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06a24 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.668524:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06a24 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26959872->26963967] (req 26959872->26963967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.668610:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493827456:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29117us (29570us total) trans 4294973901 rc 0/0 00000100:00100000:0.0:1789760777.668621:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6612 00000100:00100000:0.0:1789760777.668630:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787925438429 time 787980512444 state 0x2 ref 0 00000100:00100000:3.0:1789760777.672292:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760777.672386:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493827584 00000100:00100000:1.0:1789760777.672396:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493827584 00000100:00100000:1.0:1789760777.672450:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787925438429 time 787980512444 state 0x0 ref 1 00000100:00100000:1.0:1789760777.672466:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787980772860 time 787980512444 state 0x1 ref 1 00000100:00100000:1.0:1789760777.672496:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787980772860 time 788013531762 state 0x0 ref 1 00000100:00100000:1.0:1789760777.672508:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493827584:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760777.672552:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26963968, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd007 00010000:00010000:1.0:1789760777.672561:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06a2b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.672582:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=788s v=(0 0) 00010000:00010000:1.0:1789760777.672586:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.672592:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06a2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.672606:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06a2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.699321:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973902 is committed 00002000:00010000:2.0:1789760777.699428:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26963968 -> 26968064 00010000:00010000:2.0:1789760777.699442:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06a2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.699456:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06a2b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.699465:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06a2b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.699472:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760777.699477:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06a2b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.699486:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06a2b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.699505:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06a2b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.699513:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06a2b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26963968->26968063] (req 26963968->26968063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.699605:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493827584:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27093us (27318us total) trans 4294973902 rc 0/0 00000100:00100000:2.0:1789760777.699615:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6613 00000100:00100000:2.0:1789760777.699625:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787980772860 time 788013531762 state 0x2 ref 0 00000100:00100000:1.0:1789760777.703427:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760777.703834:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493827712 00000100:00100000:0.0:1789760777.703854:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493827712 00000100:00100000:0.0:1789760777.703916:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 787980772860 time 788013531762 state 0x0 ref 1 00000100:00100000:0.0:1789760777.703931:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788013792178 time 788013531762 state 0x1 ref 1 00000100:00100000:0.0:1789760777.704433:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788013792178 time 788045465421 state 0x0 ref 1 00000100:00100000:0.0:1789760777.704449:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493827712:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760777.704499:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26968064, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd007 00010000:00010000:0.0:1789760777.704509:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06a32 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.704525:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=788s v=(0 0) 00010000:00010000:0.0:1789760777.704529:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.704536:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06a32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.704550:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06a32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.752499:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973903 is committed 00002000:00010000:0.0:1789760777.752592:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26968064 -> 26972160 00010000:00010000:0.0:1789760777.752607:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06a32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.752621:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06a32 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.752630:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06a32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.752637:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760777.752643:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06a32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.752652:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06a32 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.752670:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06a32 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.752676:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06a32 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26968064->26972159] (req 26968064->26972159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.752918:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493827712:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48466us (49500us total) trans 4294973903 rc 0/0 00000100:00100000:0.0:1789760777.752929:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6614 00000100:00100000:0.0:1789760777.752939:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788013792178 time 788045465421 state 0x2 ref 0 00000100:00100000:2.0:1789760777.756663:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760777.756817:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493827840 00000100:00100000:0.0:1789760777.756827:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493827840 00000100:00100000:0.0:1789760777.756885:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788013792178 time 788045465421 state 0x0 ref 1 00000100:00100000:0.0:1789760777.756900:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788045725837 time 788045465421 state 0x1 ref 1 00000100:00100000:0.0:1789760777.756933:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788045725837 time 788097967452 state 0x0 ref 1 00000100:00100000:0.0:1789760777.756945:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493827840:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760777.757168:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26972160, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd007 00010000:00010000:0.0:1789760777.757178:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06a39 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.757192:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=788s v=(0 0) 00010000:00010000:0.0:1789760777.757196:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.757202:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06a39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.757226:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06a39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.786282:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760777.798977:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973904 is committed 00002000:00010000:1.0:1789760777.799070:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26972160 -> 26976256 00010000:00010000:1.0:1789760777.799085:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06a39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.799098:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06a39 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.799107:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06a39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.799115:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760777.799119:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06a39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.799128:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06a39 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.799151:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06a39 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.799158:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06a39 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26972160->26976255] (req 26972160->26976255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.799293:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493827840:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42345us (42634us total) trans 4294973904 rc 0/0 00000100:00100000:1.0:1789760777.799303:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6615 00000100:00100000:1.0:1789760777.799313:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788045725837 time 788097967452 state 0x2 ref 0 00000100:00100000:3.0:1789760777.803006:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760777.803086:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493827968 00000100:00100000:2.0:1789760777.803098:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493827968 00000100:00100000:2.0:1789760777.803155:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788045725837 time 788097967452 state 0x0 ref 1 00000100:00100000:2.0:1789760777.803170:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788098227868 time 788097967452 state 0x1 ref 1 00000100:00100000:2.0:1789760777.803198:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788098227868 time 788144233544 state 0x0 ref 1 00000100:00100000:2.0:1789760777.803222:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493827968:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760777.803268:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26976256, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd007 00010000:00010000:2.0:1789760777.803277:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06a40 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.803293:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=788s v=(0 0) 00010000:00010000:2.0:1789760777.803298:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.803304:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06a40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.803322:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06a40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.829306:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973905 is committed 00002000:00010000:1.0:1789760777.829715:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26976256 -> 26980352 00010000:00010000:1.0:1789760777.829732:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06a40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.829744:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06a40 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.829752:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06a40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.829759:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760777.829764:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06a40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.829780:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06a40 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.829799:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06a40 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.829808:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06a40 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26976256->26980351] (req 26976256->26980351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.829896:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493827968:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26682us (26893us total) trans 4294973905 rc 0/0 00000100:00100000:1.0:1789760777.829906:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6616 00000100:00100000:1.0:1789760777.829916:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788098227868 time 788144233544 state 0x2 ref 0 00000100:00100000:3.0:1789760777.834282:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760777.834704:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493828096 00000100:00100000:2.0:1789760777.834714:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493828096 00000100:00100000:2.0:1789760777.834768:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788098227868 time 788144233544 state 0x0 ref 1 00000100:00100000:2.0:1789760777.834784:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788144493960 time 788144233544 state 0x1 ref 1 00000100:00100000:2.0:1789760777.834810:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788144493960 time 788175845149 state 0x0 ref 1 00000100:00100000:2.0:1789760777.834823:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493828096:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760777.834871:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26980352, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd007 00010000:00010000:2.0:1789760777.834880:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06a47 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.834896:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=788s v=(0 0) 00010000:00010000:2.0:1789760777.834902:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.834908:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06a47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.834922:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06a47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.874092:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973906 is committed 00002000:00010000:2.0:1789760777.874192:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26980352 -> 26984448 00010000:00010000:2.0:1789760777.874217:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06a47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.874230:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06a47 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.874240:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06a47 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.874248:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760777.874252:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06a47 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.874370:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06a47 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.874390:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06a47 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.874398:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06a47 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26980352->26984447] (req 26980352->26984447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.874484:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493828096:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39659us (40206us total) trans 4294973906 rc 0/0 00000100:00100000:2.0:1789760777.874494:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6617 00000100:00100000:2.0:1789760777.874503:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788144493960 time 788175845149 state 0x2 ref 0 00000100:00100000:2.0:1789760777.879687:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760777.879894:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493828224 00000100:00100000:3.0:1789760777.879903:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493828224 00000100:00100000:3.0:1789760777.879956:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788144493960 time 788175845149 state 0x0 ref 1 00000100:00100000:3.0:1789760777.880123:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788176105565 time 788175845149 state 0x1 ref 1 00000100:00100000:3.0:1789760777.880152:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788176105565 time 788221187664 state 0x0 ref 1 00000100:00100000:3.0:1789760777.880165:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493828224:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760777.880221:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26984448, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd007 00010000:00010000:3.0:1789760777.880229:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06a4e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.880245:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=788s v=(0 0) 00010000:00010000:3.0:1789760777.880250:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.880256:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06a4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.880376:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06a4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760777.900069:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26984448 -> 26988544 00010000:00010000:3.0:1789760777.900089:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06a4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.900102:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06a4e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.900111:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06a4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.900119:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760777.900124:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06a4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.900134:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06a4e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.900153:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06a4e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.900162:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06a4e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26984448->26988543] (req 26984448->26988543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.900268:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493828224:12345-192.168.203.60@tcp:4:dd.0 Request processed in 20094us (20578us total) trans 4294973907 rc 0/0 00000100:00100000:3.0:1789760777.900280:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6618 00000100:00100000:3.0:1789760777.900290:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788176105565 time 788221187664 state 0x2 ref 0 00000001:00080000:0.0:1789760777.900887:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973907 is committed 00000100:00100000:0.0:1789760777.904385:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760777.904448:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493828352 00000100:00100000:1.0:1789760777.904458:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493828352 00000100:00100000:1.0:1789760777.904514:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788176105565 time 788221187664 state 0x0 ref 1 00000100:00100000:1.0:1789760777.904529:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788221448080 time 788221187664 state 0x1 ref 1 00000100:00100000:1.0:1789760777.904561:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788221448080 time 788245596896 state 0x0 ref 1 00000100:00100000:1.0:1789760777.904573:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493828352:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760777.904620:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26988544, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd007 00010000:00010000:1.0:1789760777.904628:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06a55 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.904644:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=788s v=(0 0) 00010000:00010000:1.0:1789760777.904648:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.904655:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06a55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.904668:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06a55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760777.928240:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26988544 -> 26992640 00010000:00010000:1.0:1789760777.928413:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06a55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.928428:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06a55 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.928437:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06a55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.928444:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760777.928449:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06a55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.928458:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06a55 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.928478:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06a55 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.928486:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06a55 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26988544->26992639] (req 26988544->26992639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.928790:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493828352:12345-192.168.203.60@tcp:4:dd.0 Request processed in 24213us (24406us total) trans 4294973908 rc 0/0 00000100:00100000:1.0:1789760777.928800:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6619 00000100:00100000:1.0:1789760777.928809:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788221448080 time 788245596896 state 0x2 ref 0 00000001:00080000:1.0:1789760777.928848:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973908 is committed 00000100:00100000:1.0:1789760777.932051:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760777.932140:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493828480 00000100:00100000:2.0:1789760777.932150:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493828480 00000100:00100000:2.0:1789760777.932213:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788221448080 time 788245596896 state 0x0 ref 1 00000100:00100000:2.0:1789760777.932229:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788245857312 time 788245596896 state 0x1 ref 1 00000100:00100000:2.0:1789760777.932258:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788245857312 time 788273293868 state 0x0 ref 1 00000100:00100000:2.0:1789760777.932279:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493828480:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760777.932326:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26992640, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd007 00010000:00010000:2.0:1789760777.932334:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a5c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.932350:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=788s v=(0 0) 00010000:00010000:2.0:1789760777.932354:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.932361:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.932374:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.962231:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973909 is committed 00002000:00010000:2.0:1789760777.962433:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26992640 -> 26996736 00010000:00010000:2.0:1789760777.962451:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.962465:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a5c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.962475:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a5c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.962482:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760777.962487:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a5c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.962496:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a5c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.962515:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a5c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.962523:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a5c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26992640->26996735] (req 26992640->26996735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.962917:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493828480:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30635us (30870us total) trans 4294973909 rc 0/0 00000100:00100000:2.0:1789760777.962932:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6620 00000100:00100000:2.0:1789760777.962942:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788245857312 time 788273293868 state 0x2 ref 0 00000100:00100000:2.0:1789760777.967544:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760777.967657:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493828608 00000100:00100000:1.0:1789760777.967668:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493828608 00000100:00100000:1.0:1789760777.967718:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788245857312 time 788273293868 state 0x0 ref 1 00000100:00100000:1.0:1789760777.967733:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788273554284 time 788273293868 state 0x1 ref 1 00000100:00100000:1.0:1789760777.967801:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788273554284 time 788308836642 state 0x0 ref 1 00000100:00100000:1.0:1789760777.967815:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493828608:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760777.967862:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 26996736, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd007 00010000:00010000:1.0:1789760777.967871:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06a63 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.967885:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=788s v=(0 0) 00010000:00010000:1.0:1789760777.967890:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.967896:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06a63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.967910:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06a63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.992586:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973910 is committed 00002000:00010000:1.0:1789760777.992677:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 26996736 -> 27000832 00010000:00010000:1.0:1789760777.992692:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06a63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.992840:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06a63 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.992851:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06a63 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.992857:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760777.992862:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06a63 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.992870:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06a63 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.992948:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06a63 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.992956:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06a63 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [26996736->27000831] (req 26996736->27000831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.993300:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493828608:12345-192.168.203.60@tcp:4:dd.0 Request processed in 25482us (25759us total) trans 4294973910 rc 0/0 00000100:00100000:1.0:1789760777.993310:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6621 00000100:00100000:1.0:1789760777.993319:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788273554284 time 788308836642 state 0x2 ref 0 00000100:00100000:0.0:1789760777.998850:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760778.000308:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493828736 00000100:00100000:3.0:1789760778.000318:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493828736 00000100:00100000:2.0:1789760778.000665:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788273554284 time 788308836642 state 0x0 ref 1 00000100:00100000:2.0:1789760778.000860:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788309097058 time 788308836642 state 0x1 ref 1 00000100:00100000:2.0:1789760778.000949:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788309097058 time 788341983721 state 0x0 ref 1 00000100:00100000:2.0:1789760778.000994:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493828736:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760778.001043:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27000832, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd007 00010000:00010000:2.0:1789760778.001058:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a6a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.001074:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=788s v=(0 0) 00010000:00010000:2.0:1789760778.001079:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.001086:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.001099:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.028714:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973911 is committed 00002000:00010000:3.0:1789760778.028828:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27000832 -> 27004928 00010000:00010000:3.0:1789760778.028854:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.028869:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a6a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.028878:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.028886:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760778.028890:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.028906:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a6a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.028926:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a6a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.028934:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a6a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27000832->27004927] (req 27000832->27004927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.029606:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493828736:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28640us (30760us total) trans 4294973911 rc 0/0 00000100:00100000:3.0:1789760778.029617:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6622 00000100:00100000:3.0:1789760778.029626:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788309097058 time 788341983721 state 0x2 ref 0 00000100:00100000:3.0:1789760778.041693:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760778.041793:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493828864 00000100:00100000:2.0:1789760778.041803:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493828864 00000100:00100000:2.0:1789760778.041860:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788309097058 time 788341983721 state 0x0 ref 1 00000100:00100000:2.0:1789760778.041877:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788342244137 time 788341983721 state 0x1 ref 1 00000100:00100000:2.0:1789760778.042160:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788342244137 time 788383193411 state 0x0 ref 1 00000100:00100000:2.0:1789760778.042175:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493828864:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760778.042236:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27004928, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd007 00010000:00010000:2.0:1789760778.042244:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06a71 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.042270:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=788s v=(0 0) 00010000:00010000:2.0:1789760778.042278:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.042284:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06a71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.042298:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06a71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760778.069155:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27004928 -> 27009024 00010000:00010000:2.0:1789760778.069175:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06a71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.069187:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06a71 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.069195:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06a71 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.069219:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760778.069223:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06a71 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.069231:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06a71 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.069249:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06a71 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.069267:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06a71 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27004928->27009023] (req 27004928->27009023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.069357:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493828864:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27180us (27667us total) trans 4294973912 rc 0/0 00000100:00100000:2.0:1789760778.069368:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6623 00000100:00100000:2.0:1789760778.069378:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788342244137 time 788383193411 state 0x2 ref 0 00000001:00080000:2.0:1789760778.070037:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973912 is committed 00000100:00100000:2.0:1789760778.076888:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760778.076989:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493828992 00000100:00100000:1.0:1789760778.076998:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493828992 00000100:00100000:1.0:1789760778.077052:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788342244137 time 788383193411 state 0x0 ref 1 00000100:00100000:1.0:1789760778.077067:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788383453827 time 788383193411 state 0x1 ref 1 00000100:00100000:1.0:1789760778.077097:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788383453827 time 788418132099 state 0x0 ref 1 00000100:00100000:1.0:1789760778.077120:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbeea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493828992:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760778.077167:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27009024, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd007 00010000:00010000:1.0:1789760778.077175:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06a78 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.077191:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=788s v=(0 0) 00010000:00010000:1.0:1789760778.077196:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.077203:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06a78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.077229:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06a78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.116016:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973913 is committed 00002000:00010000:1.0:1789760778.116085:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27009024 -> 27013120 00010000:00010000:1.0:1789760778.116099:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06a78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.116112:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06a78 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.116122:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06a78 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.116129:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760778.116134:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06a78 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.116142:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06a78 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.116160:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06a78 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.116168:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06a78 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27009024->27013119] (req 27009024->27013119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.116608:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbeea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493828992:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39483us (39723us total) trans 4294973913 rc 0/0 00000100:00100000:1.0:1789760778.116620:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6624 00000100:00100000:1.0:1789760778.116629:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788383453827 time 788418132099 state 0x2 ref 0 00000100:00100000:1.0:1789760778.120212:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760778.120373:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493829120 00000100:00100000:2.0:1789760778.120382:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493829120 00000100:00100000:2.0:1789760778.120441:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788383453827 time 788418132099 state 0x0 ref 1 00000100:00100000:2.0:1789760778.120456:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788418392515 time 788418132099 state 0x1 ref 1 00000100:00100000:2.0:1789760778.120495:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788418392515 time 788461530864 state 0x0 ref 1 00000100:00100000:2.0:1789760778.120507:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4badc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493829120:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760778.120555:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27013120, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd007 00010000:00010000:2.0:1789760778.120564:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06a7f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.120580:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=788s v=(0 0) 00010000:00010000:2.0:1789760778.120585:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.120591:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06a7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.120605:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06a7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.166080:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973914 is committed 00002000:00010000:3.0:1789760778.166172:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27013120 -> 27017216 00010000:00010000:3.0:1789760778.166188:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06a7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.166201:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06a7f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.166222:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06a7f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.166235:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760778.166240:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06a7f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.166249:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06a7f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.166280:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06a7f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.166288:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06a7f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27013120->27017215] (req 27013120->27017215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.166373:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4badc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493829120:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45864us (46175us total) trans 4294973914 rc 0/0 00000100:00100000:3.0:1789760778.166383:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6625 00000100:00100000:3.0:1789760778.166393:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788418392515 time 788461530864 state 0x2 ref 0 00000100:00100000:3.0:1789760778.171150:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760778.171428:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493829248 00000100:00100000:2.0:1789760778.171439:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493829248 00000100:00100000:2.0:1789760778.171495:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788418392515 time 788461530864 state 0x0 ref 1 00000100:00100000:2.0:1789760778.171510:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788461791280 time 788461530864 state 0x1 ref 1 00000100:00100000:2.0:1789760778.171597:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788461791280 time 788512632120 state 0x0 ref 1 00000100:00100000:2.0:1789760778.171610:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493829248:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760778.171659:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27017216, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd007 00010000:00010000:2.0:1789760778.171670:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a86 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.171685:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=788s v=(0 0) 00010000:00010000:2.0:1789760778.171689:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.171696:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.171708:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.200156:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973915 is committed 00002000:00010000:2.0:1789760778.200394:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27017216 -> 27021312 00010000:00010000:2.0:1789760778.200409:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.200420:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a86 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.200429:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a86 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.200436:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760778.200441:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a86 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.200451:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a86 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.200474:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a86 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.200482:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a86 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27017216->27021311] (req 27017216->27021311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.200642:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493829248:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29030us (29496us total) trans 4294973915 rc 0/0 00000100:00100000:2.0:1789760778.200654:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6626 00000100:00100000:2.0:1789760778.200663:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788461791280 time 788512632120 state 0x2 ref 0 00000100:00100000:1.0:1789760778.205578:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760778.205668:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493829376 00000100:00100000:0.0:1789760778.205677:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493829376 00000100:00100000:0.0:1789760778.205725:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788461791280 time 788512632120 state 0x0 ref 1 00000100:00100000:0.0:1789760778.205740:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788512892536 time 788512632120 state 0x1 ref 1 00000100:00100000:0.0:1789760778.205770:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788512892536 time 788546805540 state 0x0 ref 1 00000100:00100000:0.0:1789760778.205901:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493829376:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760778.205951:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27021312, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd007 00010000:00010000:0.0:1789760778.206046:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06a8d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.206062:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=788s v=(0 0) 00010000:00010000:0.0:1789760778.206067:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.206072:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06a8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.206096:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06a8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760778.241882:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27021312 -> 27025408 00010000:00010000:2.0:1789760778.241903:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06a8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.241915:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06a8d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.241924:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06a8d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.241932:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760778.241937:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06a8d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.241951:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06a8d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.242001:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06a8d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.242009:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06a8d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27021312->27025407] (req 27021312->27025407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.242088:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493829376:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36187us (36514us total) trans 4294973916 rc 0/0 00000100:00100000:2.0:1789760778.242098:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6627 00000100:00100000:2.0:1789760778.242108:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788512892536 time 788546805540 state 0x2 ref 0 00000001:00080000:1.0:1789760778.242178:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973916 is committed 00000100:00100000:1.0:1789760778.249106:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760778.249213:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493829504 00000100:00100000:2.0:1789760778.249222:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493829504 00000100:00100000:2.0:1789760778.249276:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788512892536 time 788546805540 state 0x0 ref 1 00000100:00100000:2.0:1789760778.249291:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788547065956 time 788546805540 state 0x1 ref 1 00000100:00100000:2.0:1789760778.249325:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788547065956 time 788590360838 state 0x0 ref 1 00000100:00100000:2.0:1789760778.249337:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493829504:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760778.249385:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27025408, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd007 00010000:00010000:2.0:1789760778.249394:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a94 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.249409:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=788s v=(0 0) 00010000:00010000:2.0:1789760778.249414:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.249420:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.249432:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.289639:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973917 is committed 00002000:00010000:2.0:1789760778.289792:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27025408 -> 27029504 00010000:00010000:2.0:1789760778.289813:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.289828:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a94 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.289839:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.289847:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760778.289853:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.289865:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a94 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.289890:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a94 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.289899:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a94 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27025408->27029503] (req 27025408->27029503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.290069:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493829504:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40726us (40965us total) trans 4294973917 rc 0/0 00000100:00100000:2.0:1789760778.290081:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6628 00000100:00100000:2.0:1789760778.290094:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788547065956 time 788590360838 state 0x2 ref 0 00000100:00100000:3.0:1789760778.295809:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760778.296626:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493829632 00000100:00100000:2.0:1789760778.296641:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493829632 00000100:00100000:2.0:1789760778.296791:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788547065956 time 788590360838 state 0x0 ref 1 00000100:00100000:2.0:1789760778.296811:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788590621254 time 788590360838 state 0x1 ref 1 00000100:00100000:2.0:1789760778.296853:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788590621254 time 788637888524 state 0x0 ref 1 00000100:00100000:2.0:1789760778.296868:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493829632:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760778.297017:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27029504, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd007 00010000:00010000:2.0:1789760778.297027:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a9b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.297048:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=788s v=(0 0) 00010000:00010000:2.0:1789760778.297053:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.297061:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.297085:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.344338:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760778.344430:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760778.344470:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760778.344524:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493830016 00000100:00100000:1.0:1789760778.344536:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493830016 00000100:00100000:1.0:1789760778.344619:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493829888 00000100:00100000:1.0:1789760778.344623:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493829888 00000100:00100000:1.0:1789760778.344655:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 517 00000100:00100000:1.0:1789760778.344667:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2367:x1876699493830016:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:1.0:1789760778.344764:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48df100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2367:x1876699493830016:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 99us (332us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760778.344773:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 517 00000100:00100000:0.0:1789760778.344784:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 518 00000100:00100000:0.0:1789760778.344799:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2367:x1876699493829888:12345-192.168.203.60@tcp:400:kworker.0 00000100:00080000:1.0:1789760778.344801:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760778.344840:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2367:x1876699493829888:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 46us (368us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760778.344847:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 518 00000100:00100000:3.0:1789760778.345037:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493829760 00000100:00100000:3.0:1789760778.345219:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493829760 00000100:00100000:3.0:1789760778.345262:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 413 00000100:00100000:3.0:1789760778.345269:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d4bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2367:x1876699493829760:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:3.0:1789760778.345313:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d4bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2367:x1876699493829760:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 44us (992us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760778.345321:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 413 00000001:00080000:3.0:1789760778.356189:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973918 is committed 00002000:00010000:0.0:1789760778.356341:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27029504 -> 27033600 00010000:00010000:0.0:1789760778.356362:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.356378:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a9b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.356388:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.356402:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760778.356407:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.356417:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a9b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.356441:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a9b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.356450:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06a9b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27029504->27033599] (req 27029504->27033599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.356582:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493829632:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59709us (61079us total) trans 4294973918 rc 0/0 00000100:00100000:0.0:1789760778.356595:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6629 00000100:00100000:0.0:1789760778.356608:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788590621254 time 788637888524 state 0x2 ref 0 00000100:00100000:0.0:1789760778.360049:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760778.360152:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493830144 00000100:00100000:2.0:1789760778.360165:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493830144 00000100:00100000:2.0:1789760778.360248:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788590621254 time 788637888524 state 0x0 ref 1 00000100:00100000:2.0:1789760778.360276:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788638148940 time 788637888524 state 0x1 ref 1 00000100:00100000:2.0:1789760778.360303:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788638148940 time 788701338731 state 0x0 ref 1 00000100:00100000:2.0:1789760778.360316:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493830144:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760778.360380:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27033600, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd007 00010000:00010000:2.0:1789760778.360390:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06aa2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.360409:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=788s v=(0 0) 00010000:00010000:2.0:1789760778.360414:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.360421:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06aa2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.360438:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06aa2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.404182:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973919 is committed 00002000:00010000:3.0:1789760778.404380:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27033600 -> 27037696 00010000:00010000:3.0:1789760778.404402:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06aa2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.404418:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06aa2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.404428:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06aa2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.404436:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760778.404441:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06aa2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.404452:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06aa2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.404477:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06aa2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.404484:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06aa2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27033600->27037695] (req 27033600->27037695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.404735:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493830144:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44413us (44693us total) trans 4294973919 rc 0/0 00000100:00100000:3.0:1789760778.404748:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6630 00000100:00100000:3.0:1789760778.404760:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788638148940 time 788701338731 state 0x2 ref 0 00000100:00100000:3.0:1789760778.407598:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760778.407714:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493830272 00000100:00100000:0.0:1789760778.407728:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493830272 00000100:00100000:0.0:1789760778.407805:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788638148940 time 788701338731 state 0x0 ref 1 00000100:00100000:0.0:1789760778.407826:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788701599147 time 788701338731 state 0x1 ref 1 00000100:00100000:0.0:1789760778.407869:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788701599147 time 788748904356 state 0x0 ref 1 00000100:00100000:0.0:1789760778.407884:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493830272:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760778.408005:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27037696, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd007 00010000:00010000:0.0:1789760778.408016:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06aa9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.408035:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=788s v=(0 0) 00010000:00010000:0.0:1789760778.408041:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.408049:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06aa9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.408066:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06aa9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.441489:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973920 is committed 00002000:00010000:1.0:1789760778.441709:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27037696 -> 27041792 00010000:00010000:1.0:1789760778.441727:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06aa9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.441739:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06aa9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.441751:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06aa9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.441756:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760778.441760:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06aa9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.441767:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06aa9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.441786:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06aa9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.441792:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06aa9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27037696->27041791] (req 27037696->27041791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.441953:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493830272:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34064us (34363us total) trans 4294973920 rc 0/0 00000100:00100000:1.0:1789760778.442014:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6631 00000100:00100000:1.0:1789760778.442027:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788701599147 time 788748904356 state 0x2 ref 0 00000100:00100000:0.0:1789760778.445901:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760778.446141:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493830400 00000100:00100000:1.0:1789760778.446156:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493830400 00000100:00100000:1.0:1789760778.446248:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788701599147 time 788748904356 state 0x0 ref 1 00000100:00100000:1.0:1789760778.446268:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788749164772 time 788748904356 state 0x1 ref 1 00000100:00100000:1.0:1789760778.446305:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788749164772 time 788787340189 state 0x0 ref 1 00000100:00100000:1.0:1789760778.446321:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c87a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493830400:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760778.446391:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27041792, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd007 00010000:00010000:1.0:1789760778.446402:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06ab0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.446420:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=788s v=(0 0) 00010000:00010000:1.0:1789760778.446425:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.446432:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06ab0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.446448:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06ab0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.480596:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973921 is committed 00002000:00010000:1.0:1789760778.480794:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27041792 -> 27045888 00010000:00010000:1.0:1789760778.480814:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06ab0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.480829:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06ab0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.480840:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06ab0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.480848:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760778.480855:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06ab0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.480866:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06ab0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.480891:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06ab0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.480900:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06ab0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27041792->27045887] (req 27041792->27045887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.481080:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c87a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493830400:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34754us (35186us total) trans 4294973921 rc 0/0 00000100:00100000:1.0:1789760778.481092:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6632 00000100:00100000:1.0:1789760778.481105:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788749164772 time 788787340189 state 0x2 ref 0 00000100:00100000:0.0:1789760778.489859:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760778.490740:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493830528 00000100:00100000:2.0:1789760778.490756:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493830528 00000100:00100000:2.0:1789760778.490837:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788749164772 time 788787340189 state 0x0 ref 1 00000100:00100000:2.0:1789760778.490906:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788787600605 time 788787340189 state 0x1 ref 1 00000100:00100000:2.0:1789760778.490940:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788787600605 time 788831976131 state 0x0 ref 1 00000100:00100000:2.0:1789760778.490955:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493830528:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760778.491036:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27045888, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd007 00010000:00010000:2.0:1789760778.491046:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06ab7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.491065:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=788s v=(0 0) 00010000:00010000:2.0:1789760778.491070:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.491076:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06ab7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.491092:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06ab7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760778.512668:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27045888 -> 27049984 00010000:00010000:0.0:1789760778.512698:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06ab7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.512715:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06ab7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.512726:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06ab7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.512739:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760778.512745:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06ab7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.512755:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06ab7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.512780:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06ab7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.512794:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06ab7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27045888->27049983] (req 27045888->27049983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.512999:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973922 is committed 00000100:00100000:0.0:1789760778.513040:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493830528:12345-192.168.203.60@tcp:4:dd.0 Request processed in 22079us (23187us total) trans 4294973922 rc 0/0 00000100:00100000:0.0:1789760778.513053:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6633 00000100:00100000:0.0:1789760778.513066:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788787600605 time 788831976131 state 0x2 ref 0 00000100:00100000:0.0:1789760778.516396:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760778.516515:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493830656 00000100:00100000:3.0:1789760778.516527:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493830656 00000100:00100000:3.0:1789760778.516596:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788787600605 time 788831976131 state 0x0 ref 1 00000100:00100000:3.0:1789760778.516614:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788832236547 time 788831976131 state 0x1 ref 1 00000100:00100000:3.0:1789760778.516803:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788832236547 time 788857836519 state 0x0 ref 1 00000100:00100000:3.0:1789760778.516820:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a37480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493830656:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760778.516888:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27049984, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd007 00010000:00010000:3.0:1789760778.516900:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06abe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.516919:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=788s v=(0 0) 00010000:00010000:3.0:1789760778.516925:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.516932:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06abe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.516954:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06abe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760778.551586:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27049984 -> 27054080 00010000:00010000:3.0:1789760778.551615:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06abe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.551630:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06abe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.551640:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06abe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.551649:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760778.551655:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06abe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.551665:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06abe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.551758:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06abe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.551768:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06abe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27049984->27054079] (req 27049984->27054079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.551892:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a37480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493830656:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35069us (35504us total) trans 4294973923 rc 0/0 00000100:00100000:3.0:1789760778.551906:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6634 00000100:00100000:3.0:1789760778.551918:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788832236547 time 788857836519 state 0x2 ref 0 00000001:00080000:1.0:1789760778.552574:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973923 is committed 00000100:00100000:1.0:1789760778.555076:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760778.555187:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493830784 00000100:00100000:0.0:1789760778.555209:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493830784 00000100:00100000:0.0:1789760778.555352:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788832236547 time 788857836519 state 0x0 ref 1 00000100:00100000:0.0:1789760778.555371:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788858096935 time 788857836519 state 0x1 ref 1 00000100:00100000:0.0:1789760778.555410:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788858096935 time 788896445543 state 0x0 ref 1 00000100:00100000:0.0:1789760778.555426:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493830784:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760778.555499:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27054080, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd007 00010000:00010000:0.0:1789760778.555511:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06ac5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.555531:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=788s v=(0 0) 00010000:00010000:0.0:1789760778.555537:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.555544:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06ac5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.555562:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06ac5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.589999:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973924 is committed 00002000:00010000:1.0:1789760778.590194:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27054080 -> 27058176 00010000:00010000:1.0:1789760778.590224:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06ac5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.590238:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06ac5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.590247:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06ac5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.590321:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760778.590327:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06ac5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.590338:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06ac5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.590361:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06ac5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.590370:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06ac5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27054080->27058175] (req 27054080->27058175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.590533:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493830784:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35103us (35464us total) trans 4294973924 rc 0/0 00000100:00100000:1.0:1789760778.590545:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6635 00000100:00100000:1.0:1789760778.590557:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788858096935 time 788896445543 state 0x2 ref 0 00000100:00100000:0.0:1789760778.594441:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760778.594553:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493830912 00000100:00100000:1.0:1789760778.594566:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493830912 00000100:00100000:1.0:1789760778.594643:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788858096935 time 788896445543 state 0x0 ref 1 00000100:00100000:1.0:1789760778.594664:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788896705959 time 788896445543 state 0x1 ref 1 00000100:00100000:1.0:1789760778.594797:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788896705959 time 788935830269 state 0x0 ref 1 00000100:00100000:1.0:1789760778.594817:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a34e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493830912:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760778.594884:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27058176, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xd007 00010000:00010000:1.0:1789760778.594895:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06acc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.594914:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=788s v=(0 0) 00010000:00010000:1.0:1789760778.594920:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.594927:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06acc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.594945:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06acc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760778.625754:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27058176 -> 27062272 00002000:00010000:1.0:1789760778.625764:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760777 -> 1789760778 00002000:00010000:1.0:1789760778.625767:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760777 -> 1789760778 00010000:00010000:1.0:1789760778.625783:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06acc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.625798:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06acc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.625808:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06acc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.625815:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760778.625821:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06acc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.625831:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06acc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.625852:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06acc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.625859:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06acc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27058176->27062271] (req 27058176->27062271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.626020:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a34e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493830912:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31199us (31586us total) trans 4294973925 rc 0/0 00000100:00100000:1.0:1789760778.626032:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6636 00000100:00100000:1.0:1789760778.626044:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788896705959 time 788935830269 state 0x2 ref 0 00000001:00080000:3.0:1789760778.627404:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973925 is committed 00000100:00100000:0.0:1789760778.628766:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760778.628937:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493831040 00000100:00100000:1.0:1789760778.628949:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493831040 00000100:00100000:1.0:1789760778.629058:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788896705959 time 788935830269 state 0x0 ref 1 00000100:00100000:1.0:1789760778.629077:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788936090685 time 788935830269 state 0x1 ref 1 00000100:00100000:1.0:1789760778.629109:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788936090685 time 788970143869 state 0x0 ref 1 00000100:00100000:1.0:1789760778.629124:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493831040:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760778.629193:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27062272, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd007 00010000:00010000:1.0:1789760778.629213:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ad3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.629233:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=788s v=(0 0) 00010000:00010000:1.0:1789760778.629239:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.629248:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ad3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.629376:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ad3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.669551:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973926 is committed 00002000:00010000:1.0:1789760778.669762:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27062272 -> 27066368 00010000:00010000:1.0:1789760778.669781:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ad3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.669796:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ad3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.669807:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ad3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.669815:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760778.669820:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ad3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.669829:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ad3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.669861:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ad3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.669937:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ad3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27062272->27066367] (req 27062272->27066367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.670346:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493831040:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41216us (41584us total) trans 4294973926 rc 0/0 00000100:00100000:1.0:1789760778.670358:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6637 00000100:00100000:1.0:1789760778.670371:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788936090685 time 788970143869 state 0x2 ref 0 00000100:00100000:2.0:1789760778.675230:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760778.675772:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493831168 00000100:00100000:3.0:1789760778.675786:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493831168 00000100:00100000:3.0:1789760778.675862:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788936090685 time 788970143869 state 0x0 ref 1 00000100:00100000:3.0:1789760778.675955:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788970404285 time 788970143869 state 0x1 ref 1 00000100:00100000:3.0:1789760778.676116:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788970404285 time 789017151327 state 0x0 ref 1 00000100:00100000:3.0:1789760778.676215:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb14850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493831168:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760778.676362:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27066368, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd007 00010000:00010000:3.0:1789760778.676374:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06ada lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.676394:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=789s v=(0 0) 00010000:00010000:3.0:1789760778.676399:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.676407:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06ada lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.676431:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06ada lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760778.716929:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27066368 -> 27070464 00010000:00010000:0.0:1789760778.716956:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06ada lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.717005:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06ada lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.717015:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06ada lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.717024:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760778.717030:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06ada lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.717041:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06ada lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.717065:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06ada lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.717074:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06ada lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27066368->27070463] (req 27066368->27070463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.717547:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb14850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493831168:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41409us (42325us total) trans 4294973927 rc 0/0 00000100:00100000:0.0:1789760778.717560:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6638 00000100:00100000:0.0:1789760778.717572:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788970404285 time 789017151327 state 0x2 ref 0 00000001:00080000:1.0:1789760778.718454:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973927 is committed 00000100:00100000:0.0:1789760778.720778:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760778.720957:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493831296 00000100:00100000:1.0:1789760778.721044:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493831296 00000100:00100000:1.0:1789760778.721227:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 788970404285 time 789017151327 state 0x0 ref 1 00000100:00100000:1.0:1789760778.721248:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789017411743 time 789017151327 state 0x1 ref 1 00000100:00100000:1.0:1789760778.721363:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789017411743 time 789062397589 state 0x0 ref 1 00000100:00100000:1.0:1789760778.721379:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493831296:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760778.721442:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27070464, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd007 00010000:00010000:1.0:1789760778.721451:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ae1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.721472:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=789s v=(0 0) 00010000:00010000:1.0:1789760778.721476:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.721481:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ae1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.721493:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ae1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.760383:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973928 is committed 00002000:00010000:1.0:1789760778.760396:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27070464 -> 27074560 00010000:00010000:1.0:1789760778.760414:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ae1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.760431:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ae1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.760445:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ae1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.760454:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760778.760460:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ae1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.760471:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ae1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.760497:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ae1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.760506:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ae1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27070464->27074559] (req 27070464->27074559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.760626:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493831296:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39244us (39857us total) trans 4294973928 rc 0/0 00000100:00100000:1.0:1789760778.760639:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6639 00000100:00100000:1.0:1789760778.760652:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789017411743 time 789062397589 state 0x2 ref 0 00000100:00100000:2.0:1789760778.763791:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760778.763991:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493831424 00000100:00100000:1.0:1789760778.764003:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493831424 00000100:00100000:1.0:1789760778.764074:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789017411743 time 789062397589 state 0x0 ref 1 00000100:00100000:1.0:1789760778.764090:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789062658005 time 789062397589 state 0x1 ref 1 00000100:00100000:1.0:1789760778.764109:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789062658005 time 789105145399 state 0x0 ref 1 00000100:00100000:1.0:1789760778.764119:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493831424:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760778.764379:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27074560, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd007 00010000:00010000:1.0:1789760778.764390:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06ae8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.764410:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=789s v=(0 0) 00010000:00010000:1.0:1789760778.764415:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.764422:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06ae8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.764444:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06ae8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760778.805135:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27074560 -> 27078656 00010000:00010000:1.0:1789760778.805164:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06ae8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.805181:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06ae8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.805191:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06ae8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.805194:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973929 is committed 00010000:00010000:1.0:1789760778.805219:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760778.805225:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06ae8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.805236:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06ae8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.805366:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06ae8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.805377:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06ae8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27074560->27078655] (req 27074560->27078655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.805556:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493831424:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41429us (41772us total) trans 4294973929 rc 0/0 00000100:00100000:1.0:1789760778.805570:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6640 00000100:00100000:1.0:1789760778.805581:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789062658005 time 789105145399 state 0x2 ref 0 00000100:00100000:2.0:1789760778.810115:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760778.810424:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493831552 00000100:00100000:0.0:1789760778.810437:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493831552 00000100:00100000:0.0:1789760778.810510:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789062658005 time 789105145399 state 0x0 ref 1 00000100:00100000:0.0:1789760778.810529:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789105405815 time 789105145399 state 0x1 ref 1 00000100:00100000:0.0:1789760778.810558:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789105405815 time 789151594109 state 0x0 ref 1 00000100:00100000:0.0:1789760778.810571:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493831552:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760778.810633:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27078656, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd007 00010000:00010000:0.0:1789760778.810643:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06aef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.810664:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=789s v=(0 0) 00010000:00010000:0.0:1789760778.810669:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.810677:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06aef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.810693:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06aef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.811904:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760778.854600:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27078656 -> 27082752 00010000:00010000:3.0:1789760778.854629:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06aef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.854645:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06aef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.854656:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06aef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.854665:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760778.854671:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06aef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.854681:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06aef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.854706:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06aef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.854715:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06aef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27078656->27082751] (req 27078656->27082751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.854840:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493831552:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44264us (44733us total) trans 4294973930 rc 0/0 00000100:00100000:3.0:1789760778.854852:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6641 00000100:00100000:3.0:1789760778.854864:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789105405815 time 789151594109 state 0x2 ref 0 00000001:00080000:0.0:1789760778.855793:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973930 is committed 00000100:00100000:0.0:1789760778.859473:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760778.859554:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493831680 00000100:00100000:3.0:1789760778.859568:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493831680 00000100:00100000:3.0:1789760778.859642:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789105405815 time 789151594109 state 0x0 ref 1 00000100:00100000:3.0:1789760778.859662:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789151854525 time 789151594109 state 0x1 ref 1 00000100:00100000:3.0:1789760778.859702:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789151854525 time 789200737591 state 0x0 ref 1 00000100:00100000:3.0:1789760778.859716:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a37480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493831680:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760778.859786:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27082752, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd007 00010000:00010000:3.0:1789760778.859798:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06af6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.859819:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=789s v=(0 0) 00010000:00010000:3.0:1789760778.859824:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.859832:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06af6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.859849:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06af6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.896691:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973931 is committed 00002000:00010000:3.0:1789760778.896803:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27082752 -> 27086848 00010000:00010000:3.0:1789760778.896823:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06af6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.896838:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06af6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.896847:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06af6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.896855:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760778.896860:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06af6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.896869:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06af6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.896893:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06af6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.896901:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06af6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27082752->27086847] (req 27082752->27086847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.897164:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a37480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493831680:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37442us (37698us total) trans 4294973931 rc 0/0 00000100:00100000:3.0:1789760778.897180:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6642 00000100:00100000:3.0:1789760778.897194:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789151854525 time 789200737591 state 0x2 ref 0 00000100:00100000:1.0:1789760778.900915:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760778.901366:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493831808 00000100:00100000:0.0:1789760778.901380:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493831808 00000100:00100000:0.0:1789760778.901456:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789151854525 time 789200737591 state 0x0 ref 1 00000100:00100000:0.0:1789760778.901475:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789200998007 time 789200737591 state 0x1 ref 1 00000100:00100000:0.0:1789760778.901740:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789200998007 time 789242772909 state 0x0 ref 1 00000100:00100000:0.0:1789760778.901756:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493831808:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760778.901823:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27086848, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd007 00010000:00010000:0.0:1789760778.901834:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06afd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.901860:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=789s v=(0 0) 00010000:00010000:0.0:1789760778.901865:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.901872:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06afd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.901888:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06afd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.936234:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973932 is committed 00002000:00010000:0.0:1789760778.936369:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27086848 -> 27090944 00010000:00010000:0.0:1789760778.936391:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06afd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.936409:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06afd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.936419:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06afd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.936427:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760778.936433:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06afd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.936444:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06afd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.936470:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06afd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.936480:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06afd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27086848->27090943] (req 27086848->27090943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.936606:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493831808:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34846us (35697us total) trans 4294973932 rc 0/0 00000100:00100000:0.0:1789760778.936619:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6643 00000100:00100000:0.0:1789760778.936631:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789200998007 time 789242772909 state 0x2 ref 0 00000100:00100000:1.0:1789760778.940925:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760778.941185:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493831936 00000100:00100000:2.0:1789760778.941207:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493831936 00000100:00100000:2.0:1789760778.941495:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789200998007 time 789242772909 state 0x0 ref 1 00000100:00100000:2.0:1789760778.941515:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789243033325 time 789242772909 state 0x1 ref 1 00000100:00100000:2.0:1789760778.941553:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789243033325 time 789282587953 state 0x0 ref 1 00000100:00100000:2.0:1789760778.941569:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493831936:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760778.941640:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27090944, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd007 00010000:00010000:2.0:1789760778.941650:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06b04 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.941669:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=789s v=(0 0) 00010000:00010000:2.0:1789760778.941674:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.941870:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06b04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.941888:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06b04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.986477:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973933 is committed 00002000:00010000:3.0:1789760778.986581:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27090944 -> 27095040 00010000:00010000:3.0:1789760778.986600:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06b04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.986615:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06b04 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.986625:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06b04 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.986633:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760778.986639:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06b04 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.986650:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06b04 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.986672:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06b04 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.986679:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06b04 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27090944->27095039] (req 27090944->27095039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.986802:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493831936:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45232us (45887us total) trans 4294973933 rc 0/0 00000100:00100000:3.0:1789760778.986813:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6644 00000100:00100000:3.0:1789760778.986825:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789243033325 time 789282587953 state 0x2 ref 0 00000100:00100000:2.0:1789760778.990131:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760778.990272:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493832064 00000100:00100000:3.0:1789760778.990286:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493832064 00000100:00100000:3.0:1789760778.990360:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789243033325 time 789282587953 state 0x0 ref 1 00000100:00100000:3.0:1789760778.990379:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789282848369 time 789282587953 state 0x1 ref 1 00000100:00100000:3.0:1789760778.990405:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789282848369 time 789331440061 state 0x0 ref 1 00000100:00100000:3.0:1789760778.990419:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493832064:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760778.990492:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27095040, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd007 00010000:00010000:3.0:1789760778.990503:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06b0b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.990525:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=789s v=(0 0) 00010000:00010000:3.0:1789760778.990530:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.990538:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06b0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.990555:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06b0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.041100:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973934 is committed 00002000:00010000:0.0:1789760779.041206:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27095040 -> 27099136 00010000:00010000:0.0:1789760779.041225:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06b0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.041241:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06b0b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.041250:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06b0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.041267:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760779.041272:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06b0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.041282:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06b0b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.041314:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06b0b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.041322:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06b0b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27095040->27099135] (req 27095040->27099135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.041457:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493832064:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51033us (51334us total) trans 4294973934 rc 0/0 00000100:00100000:0.0:1789760779.041470:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6645 00000100:00100000:0.0:1789760779.041481:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789282848369 time 789331440061 state 0x2 ref 0 00000100:00100000:1.0:1789760779.050461:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760779.050692:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493832192 00000100:00100000:3.0:1789760779.050705:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493832192 00000100:00100000:3.0:1789760779.050797:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789282848369 time 789331440061 state 0x0 ref 1 00000100:00100000:3.0:1789760779.050817:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789331700477 time 789331440061 state 0x1 ref 1 00000100:00100000:3.0:1789760779.050856:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789331700477 time 789391891016 state 0x0 ref 1 00000100:00100000:3.0:1789760779.050871:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493832192:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760779.050935:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27099136, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd007 00010000:00010000:3.0:1789760779.050946:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06b12 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.051013:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=789s v=(0 0) 00010000:00010000:3.0:1789760779.051020:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.051027:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06b12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.051043:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06b12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760779.051833:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789760779.092879:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27099136 -> 27103232 00000001:00080000:2.0:1789760779.092892:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973935 is committed 00010000:00010000:1.0:1789760779.092908:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06b12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.092923:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06b12 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.092933:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06b12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.092941:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760779.092946:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06b12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.093677:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06b12 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.093776:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06b12 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.093786:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06b12 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27099136->27103231] (req 27099136->27103231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.093929:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493832192:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43053us (43490us total) trans 4294973935 rc 0/0 00000100:00100000:1.0:1789760779.093943:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6646 00000100:00100000:1.0:1789760779.093956:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789331700477 time 789391891016 state 0x2 ref 0 00000100:00100000:1.0:1789760779.100347:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760779.100844:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493832320 00000100:00100000:0.0:1789760779.100857:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493832320 00000100:00100000:0.0:1789760779.100935:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789331700477 time 789391891016 state 0x0 ref 1 00000100:00100000:0.0:1789760779.100955:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789392151432 time 789391891016 state 0x1 ref 1 00000100:00100000:3.0:1789760779.101409:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789392151432 time 789442440185 state 0x0 ref 1 00000100:00100000:3.0:1789760779.101431:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493832320:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760779.101505:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27103232, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd007 00010000:00010000:3.0:1789760779.101518:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b19 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.101541:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=789s v=(0 0) 00010000:00010000:3.0:1789760779.101546:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.101554:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.101571:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.130229:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760779.130268:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc48dfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3303:1876699519259648:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760779.133519:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760779.133698:0:6691:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519259648 00000100:00100000:3.0:1789760779.133708:0:6691:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519259648 00000100:00100000:3.0:1789760779.133855:0:6691:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 214 00000100:00100000:3.0:1789760779.133869:0:6691:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876699519259648:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789760779.133896:0:6691:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528547840 pending 1703936 free 3831496704 avail 3819962368 00000100:00100000:3.0:1789760779.133990:0:6691:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbec700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876699519259648:12345-0@lo:13:osp-pre-0-0.0 Request processed in 125us (478us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760779.133998:0:6691:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 214 00000100:00100000:3.0:1789760779.134053:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc48dfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3303:1876699519259648:0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1789760779.134222:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760779.163945:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973936 is committed 00002000:00010000:3.0:1789760779.164145:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27103232 -> 27107328 00010000:00010000:3.0:1789760779.164171:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.164184:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b19 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.164194:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.164212:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760779.164216:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.164225:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b19 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.164245:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b19 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.164362:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b19 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27103232->27107327] (req 27103232->27107327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.164456:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493832320:12345-192.168.203.60@tcp:4:dd.0 Request processed in 63028us (64120us total) trans 4294973936 rc 0/0 00000100:00100000:3.0:1789760779.164468:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6647 00000100:00100000:3.0:1789760779.164477:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789392151432 time 789442440185 state 0x2 ref 0 00000100:00100000:2.0:1789760779.167404:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760779.167482:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493832448 00000100:00100000:3.0:1789760779.167500:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493832448 00000100:00100000:3.0:1789760779.167578:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789392151432 time 789442440185 state 0x0 ref 1 00000100:00100000:3.0:1789760779.167599:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789442700601 time 789442440185 state 0x1 ref 1 00000100:00100000:3.0:1789760779.167806:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789442700601 time 789508840217 state 0x0 ref 1 00000100:00100000:3.0:1789760779.167823:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbedf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493832448:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760779.168325:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27107328, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd007 00010000:00010000:3.0:1789760779.168336:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b20 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.168357:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=789s v=(0 0) 00010000:00010000:3.0:1789760779.168361:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.168369:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.168385:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760779.218509:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27107328 -> 27111424 00010000:00010000:3.0:1789760779.218535:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.218550:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b20 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.218560:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.218567:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760779.218574:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.218584:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b20 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.218607:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b20 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.218615:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b20 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27107328->27111423] (req 27107328->27111423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.218749:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbedf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493832448:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50922us (51347us total) trans 4294973937 rc 0/0 00000100:00100000:3.0:1789760779.218762:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6648 00000100:00100000:3.0:1789760779.218775:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789442700601 time 789508840217 state 0x2 ref 0 00000001:00080000:2.0:1789760779.219097:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973937 is committed 00000100:00100000:2.0:1789760779.222650:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760779.222725:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493832576 00000100:00100000:1.0:1789760779.222739:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493832576 00000100:00100000:1.0:1789760779.222815:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789442700601 time 789508840217 state 0x0 ref 1 00000100:00100000:1.0:1789760779.222834:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789509100633 time 789508840217 state 0x1 ref 1 00000100:00100000:1.0:1789760779.222929:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789509100633 time 789563962981 state 0x0 ref 1 00000100:00100000:1.0:1789760779.222944:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493832576:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760779.223050:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27111424, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd007 00010000:00010000:1.0:1789760779.223061:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06b27 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.223084:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=789s v=(0 0) 00010000:00010000:1.0:1789760779.223089:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.223097:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06b27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.223113:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06b27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.260406:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973938 is committed 00002000:00010000:2.0:1789760779.260482:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27111424 -> 27115520 00010000:00010000:2.0:1789760779.260497:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06b27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.260509:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06b27 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.260518:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06b27 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.260524:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760779.260529:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06b27 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.260538:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06b27 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.260557:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06b27 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.260565:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06b27 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27111424->27115519] (req 27111424->27115519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.260667:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493832576:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37720us (38025us total) trans 4294973938 rc 0/0 00000100:00100000:2.0:1789760779.260678:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6649 00000100:00100000:2.0:1789760779.260686:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789509100633 time 789563962981 state 0x2 ref 0 00000100:00100000:0.0:1789760779.265857:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760779.265955:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493832704 00000100:00100000:1.0:1789760779.265995:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493832704 00000100:00100000:1.0:1789760779.266053:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789509100633 time 789563962981 state 0x0 ref 1 00000100:00100000:1.0:1789760779.266069:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789564223397 time 789563962981 state 0x1 ref 1 00000100:00100000:3.0:1789760779.266187:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789564223397 time 789607209471 state 0x0 ref 1 00000100:00100000:3.0:1789760779.266226:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a34e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493832704:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760779.266309:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27115520, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd007 00010000:00010000:3.0:1789760779.266321:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b2e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.266340:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=789s v=(0 0) 00010000:00010000:3.0:1789760779.266346:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.266354:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.266370:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760779.303126:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27115520 -> 27119616 00000001:00080000:2.0:1789760779.303128:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973939 is committed 00010000:00010000:3.0:1789760779.303145:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.303155:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b2e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.303162:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.303167:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760779.303171:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.303177:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b2e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.303193:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b2e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.303207:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b2e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27115520->27119615] (req 27115520->27119615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.303427:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a34e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493832704:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37209us (37575us total) trans 4294973939 rc 0/0 00000100:00100000:3.0:1789760779.303436:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6650 00000100:00100000:3.0:1789760779.303444:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789564223397 time 789607209471 state 0x2 ref 0 00000100:00100000:0.0:1789760779.308449:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760779.308498:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493832832 00000100:00100000:3.0:1789760779.308509:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493832832 00000100:00100000:3.0:1789760779.308624:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789564223397 time 789607209471 state 0x0 ref 1 00000100:00100000:3.0:1789760779.308639:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789607469887 time 789607209471 state 0x1 ref 1 00000100:00100000:3.0:1789760779.308672:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789607469887 time 789649707322 state 0x0 ref 1 00000100:00100000:3.0:1789760779.308685:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493832832:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760779.308736:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27119616, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd007 00010000:00010000:3.0:1789760779.308744:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b35 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.308757:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=789s v=(0 0) 00010000:00010000:3.0:1789760779.308762:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.308769:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.308782:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.340640:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973940 is committed 00002000:00010000:3.0:1789760779.340734:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27119616 -> 27123712 00010000:00010000:3.0:1789760779.340750:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.340836:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b35 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.340844:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b35 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.340851:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760779.340855:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b35 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.340864:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b35 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.340884:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b35 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.340891:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b35 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27119616->27123711] (req 27119616->27123711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.341218:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493832832:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32530us (32772us total) trans 4294973940 rc 0/0 00000100:00100000:3.0:1789760779.341234:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6651 00000100:00100000:3.0:1789760779.341245:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789607469887 time 789649707322 state 0x2 ref 0 00000100:00100000:2.0:1789760779.357903:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760779.359874:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493832960 00000100:00100000:1.0:1789760779.359889:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493832960 00000100:00100000:1.0:1789760779.359950:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789607469887 time 789649707322 state 0x0 ref 1 00000100:00100000:1.0:1789760779.360072:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789649967738 time 789649707322 state 0x1 ref 1 00000100:00100000:1.0:1789760779.360113:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789649967738 time 789701148687 state 0x0 ref 1 00000100:00100000:1.0:1789760779.360128:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493832960:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760779.360179:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27123712, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd007 00010000:00010000:1.0:1789760779.360189:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06b3c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.360213:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=789s v=(0 0) 00010000:00010000:1.0:1789760779.360218:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.360223:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06b3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.360238:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06b3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.395036:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973941 is committed 00002000:00010000:1.0:1789760779.395145:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27123712 -> 27127808 00010000:00010000:1.0:1789760779.395166:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06b3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.395181:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06b3c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.395207:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06b3c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.395216:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760779.395222:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06b3c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.395233:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06b3c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.395332:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06b3c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.395342:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06b3c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27123712->27127807] (req 27123712->27127807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.395513:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493832960:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35382us (37613us total) trans 4294973941 rc 0/0 00000100:00100000:1.0:1789760779.395526:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6652 00000100:00100000:1.0:1789760779.395538:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789649967738 time 789701148687 state 0x2 ref 0 00000100:00100000:1.0:1789760779.402878:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760779.403025:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493833088 00000100:00100000:2.0:1789760779.403039:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493833088 00000100:00100000:2.0:1789760779.403116:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789649967738 time 789701148687 state 0x0 ref 1 00000100:00100000:2.0:1789760779.403135:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789701409103 time 789701148687 state 0x1 ref 1 00000100:00100000:2.0:1789760779.403169:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789701409103 time 789744204308 state 0x0 ref 1 00000100:00100000:2.0:1789760779.403185:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493833088:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760779.403261:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27127808, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd007 00010000:00010000:2.0:1789760779.403272:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06b43 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.403292:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=789s v=(0 0) 00010000:00010000:2.0:1789760779.403297:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.403305:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06b43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.403321:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06b43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760779.449485:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27127808 -> 27131904 00010000:00010000:0.0:1789760779.449516:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06b43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.449533:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06b43 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.449550:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06b43 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.449559:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760779.449565:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06b43 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.449576:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06b43 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.449611:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06b43 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.449620:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06b43 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27127808->27131903] (req 27127808->27131903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.449864:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493833088:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46675us (46993us total) trans 4294973942 rc 0/0 00000100:00100000:0.0:1789760779.449877:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6653 00000100:00100000:0.0:1789760779.449889:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789701409103 time 789744204308 state 0x2 ref 0 00000001:00080000:1.0:1789760779.450836:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973942 is committed 00000100:00100000:0.0:1789760779.456307:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760779.456458:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493833216 00000100:00100000:3.0:1789760779.456471:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493833216 00000100:00100000:3.0:1789760779.456547:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789701409103 time 789744204308 state 0x0 ref 1 00000100:00100000:3.0:1789760779.456567:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789744464724 time 789744204308 state 0x1 ref 1 00000100:00100000:3.0:1789760779.456604:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789744464724 time 789797639383 state 0x0 ref 1 00000100:00100000:3.0:1789760779.456623:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493833216:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760779.456702:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27131904, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd007 00010000:00010000:3.0:1789760779.456720:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b4a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.456741:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=789s v=(0 0) 00010000:00010000:3.0:1789760779.456747:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.456755:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.456771:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.506403:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973943 is committed 00002000:00010000:0.0:1789760779.506564:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27131904 -> 27136000 00010000:00010000:0.0:1789760779.506585:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.506608:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b4a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.506618:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.506627:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760779.506632:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.506643:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b4a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.506669:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b4a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.506804:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06b4a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27131904->27135999] (req 27131904->27135999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.507012:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493833216:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50383us (50712us total) trans 4294973943 rc 0/0 00000100:00100000:0.0:1789760779.507025:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6654 00000100:00100000:0.0:1789760779.507038:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789744464724 time 789797639383 state 0x2 ref 0 00000100:00100000:2.0:1789760779.515548:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760779.515653:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493833344 00000100:00100000:3.0:1789760779.515667:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493833344 00000100:00100000:3.0:1789760779.515821:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789744464724 time 789797639383 state 0x0 ref 1 00000100:00100000:3.0:1789760779.515841:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789797899799 time 789797639383 state 0x1 ref 1 00000100:00100000:3.0:1789760779.515928:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789797899799 time 789856961890 state 0x0 ref 1 00000100:00100000:3.0:1789760779.515944:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbeea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493833344:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760779.516061:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27136000, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd007 00010000:00010000:3.0:1789760779.516071:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b51 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.516097:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=789s v=(0 0) 00010000:00010000:3.0:1789760779.516103:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.516111:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.516129:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.576553:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973944 is committed 00002000:00010000:2.0:1789760779.576663:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27136000 -> 27140096 00010000:00010000:2.0:1789760779.576802:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.576820:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b51 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.576831:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b51 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.576839:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760779.576844:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b51 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.576855:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b51 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.576879:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b51 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.576887:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06b51 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27136000->27140095] (req 27136000->27140095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.577117:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbeea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493833344:12345-192.168.203.60@tcp:4:dd.0 Request processed in 61161us (61569us total) trans 4294973944 rc 0/0 00000100:00100000:2.0:1789760779.577132:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6655 00000100:00100000:2.0:1789760779.577146:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789797899799 time 789856961890 state 0x2 ref 0 00000100:00100000:1.0:1789760779.584233:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760779.584450:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493833472 00000100:00100000:3.0:1789760779.584464:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493833472 00000100:00100000:3.0:1789760779.584603:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789797899799 time 789856961890 state 0x0 ref 1 00000100:00100000:3.0:1789760779.584623:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789857222306 time 789856961890 state 0x1 ref 1 00000100:00100000:3.0:1789760779.584703:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789857222306 time 789925737527 state 0x0 ref 1 00000100:00100000:3.0:1789760779.584728:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493833472:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760779.584797:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27140096, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xd007 00010000:00010000:3.0:1789760779.584808:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b58 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.584826:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=789s v=(0 0) 00010000:00010000:3.0:1789760779.584831:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.584850:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.584867:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.622442:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973945 is committed 00002000:00010000:0.0:1789760779.622550:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27140096 -> 27144192 00002000:00010000:0.0:1789760779.622557:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760778 -> 1789760779 00002000:00010000:0.0:1789760779.622560:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760778 -> 1789760779 00010000:00010000:0.0:1789760779.622580:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.622594:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b58 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.622604:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.622613:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760779.622619:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.622629:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b58 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.622653:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b58 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.622658:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b58 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27140096->27144191] (req 27140096->27144191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.622827:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493833472:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38098us (38604us total) trans 4294973945 rc 0/0 00000100:00100000:0.0:1789760779.622837:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6656 00000100:00100000:0.0:1789760779.622849:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789857222306 time 789925737527 state 0x2 ref 0 00000100:00100000:1.0:1789760779.627249:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760779.627345:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493833600 00000100:00100000:0.0:1789760779.627358:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493833600 00000100:00100000:0.0:1789760779.627438:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789857222306 time 789925737527 state 0x0 ref 1 00000100:00100000:0.0:1789760779.627457:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789925997943 time 789925737527 state 0x1 ref 1 00000100:00100000:0.0:1789760779.627493:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789925997943 time 789968527974 state 0x0 ref 1 00000100:00100000:0.0:1789760779.627578:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4361500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493833600:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760779.627648:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27144192, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd007 00010000:00010000:0.0:1789760779.627659:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06b5f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.627680:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=789s v=(0 0) 00010000:00010000:0.0:1789760779.627685:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.627693:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06b5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.627722:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06b5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.644053:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620fb461c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3306:1876699519259904:0@lo:400:kworker.0 00000100:00100000:3.0:1789760779.644152:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760779.644242:0:3306:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1789760779.644305:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620fb460a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3306:1876699519259776:0@lo:400:kworker.0 00000100:00100000:3.0:1789760779.644341:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760779.644367:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc1d08e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519260288:0@lo:400:kworker.0 00000100:00100000:3.0:1789760779.644394:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760779.644422:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4ba7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3303:1876699519260032:0@lo:400:kworker.0 00000100:00100000:3.0:1789760779.644447:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760779.644466:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760779.644471:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4ba6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3303:1876699519260160:0@lo:400:kworker.0 00000100:00100000:3.0:1789760779.644491:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760779.644524:0:10519:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519259904 00000100:00100000:3.0:1789760779.644533:0:10519:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519259904 00000100:00100000:3.0:1789760779.644585:0:10519:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519260032 00000100:00100000:3.0:1789760779.644588:0:10519:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519260032 00000100:00100000:3.0:1789760779.644618:0:10519:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519260160 00000100:00100000:3.0:1789760779.644621:0:10519:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519260160 00000100:00100000:3.0:1789760779.644642:0:10519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 519 00000100:00100000:3.0:1789760779.644651:0:10519:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3306:x1876699519259904:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760779.644709:0:10519:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3306:x1876699519259904:12345-0@lo:400:kworker.0 Request processed in 59us (558us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760779.644717:0:10519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 519 00000100:00100000:3.0:1789760779.644736:0:10519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 520 00000100:00100000:3.0:1789760779.644741:0:10519:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876699519260032:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760779.644770:0:10519:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876699519260032:12345-0@lo:400:kworker.0 Request processed in 28us (322us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760779.644775:0:10519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 520 00000100:00100000:3.0:1789760779.644786:0:10519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 521 00000100:00100000:3.0:1789760779.644791:0:10519:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33edf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876699519260160:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760779.644816:0:10519:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33edf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876699519260160:12345-0@lo:400:kworker.0 Request processed in 25us (325us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760779.644822:0:10519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 521 00000100:00100000:3.0:1789760779.644839:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519259776 00000100:00100000:3.0:1789760779.644842:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519259776 00000100:00100000:3.0:1789760779.644935:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 414 00000100:00100000:3.0:1789760779.644944:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d49450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3306:x1876699519259776:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760779.645001:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d49450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3306:x1876699519259776:12345-0@lo:400:kworker.0 Request processed in 57us (657us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760779.645008:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 414 00000100:00100000:3.0:1789760779.645039:0:10475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519260288 00000100:00100000:3.0:1789760779.645043:0:10475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519260288 00000100:00100000:3.0:1789760779.645083:0:10475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 185 00000100:00100000:3.0:1789760779.645090:0:10475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519260288:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760779.645123:0:10475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519260288:12345-0@lo:400:kworker.0 Request processed in 34us (729us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760779.645129:0:10475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 185 00000100:00080000:3.0:1789760779.645162:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789760779.645168:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760779.645215:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620fb460a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3306:1876699519259776:0@lo:400:kworker.0 00000100:00100000:3.0:1789760779.645230:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc1d08e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519260288:0@lo:400:kworker.0 00000100:00100000:3.0:1789760779.645240:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620fb461c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3306:1876699519259904:0@lo:400:kworker.0 00000100:00100000:3.0:1789760779.645342:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4ba6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3303:1876699519260160:0@lo:400:kworker.0 00000100:00100000:3.0:1789760779.645373:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4ba7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3303:1876699519260032:0@lo:400:kworker.0 00002000:00010000:0.0:1789760779.656313:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27144192 -> 27148288 00010000:00010000:0.0:1789760779.656342:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06b5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.656363:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06b5f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.656374:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06b5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.656383:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760779.656388:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06b5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.656398:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06b5f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.656421:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06b5f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.656426:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06b5f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27144192->27148287] (req 27144192->27148287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.656533:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4361500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493833600:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28954us (29291us total) trans 4294973946 rc 0/0 00000100:00100000:0.0:1789760779.656542:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6657 00000100:00100000:0.0:1789760779.656552:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789925997943 time 789968527974 state 0x2 ref 0 00000001:00080000:2.0:1789760779.657357:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973946 is committed 00000100:00100000:2.0:1789760779.659893:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760779.660016:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493833728 00000100:00100000:0.0:1789760779.660031:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493833728 00000100:00100000:0.0:1789760779.660106:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789925997943 time 789968527974 state 0x0 ref 1 00000100:00100000:0.0:1789760779.660127:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789968788390 time 789968527974 state 0x1 ref 1 00000100:00100000:0.0:1789760779.660215:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789968788390 time 790001250097 state 0x0 ref 1 00000100:00100000:0.0:1789760779.660231:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620f39f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493833728:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760779.660298:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27148288, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd007 00010000:00010000:0.0:1789760779.660309:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06b66 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.660340:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=790s v=(0 0) 00010000:00010000:0.0:1789760779.660345:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.660352:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06b66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.660370:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06b66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.688555:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973947 is committed 00002000:00010000:0.0:1789760779.688713:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27148288 -> 27152384 00010000:00010000:0.0:1789760779.688734:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06b66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.688751:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06b66 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.688767:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06b66 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.688776:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760779.688782:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06b66 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.688794:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06b66 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.688820:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06b66 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.688829:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06b66 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27148288->27152383] (req 27148288->27152383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.688953:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620f39f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493833728:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28717us (29137us total) trans 4294973947 rc 0/0 00000100:00100000:0.0:1789760779.689011:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6658 00000100:00100000:0.0:1789760779.689025:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789968788390 time 790001250097 state 0x2 ref 0 00000100:00100000:3.0:1789760779.692387:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760779.692505:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493833856 00000100:00100000:2.0:1789760779.692517:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493833856 00000100:00100000:2.0:1789760779.692589:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 789968788390 time 790001250097 state 0x0 ref 1 00000100:00100000:2.0:1789760779.692609:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790001510513 time 790001250097 state 0x1 ref 1 00000100:00100000:2.0:1789760779.692673:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790001510513 time 790033707768 state 0x0 ref 1 00000100:00100000:2.0:1789760779.692687:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493833856:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760779.692751:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27152384, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd007 00010000:00010000:2.0:1789760779.692762:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b6d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.692780:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=790s v=(0 0) 00010000:00010000:2.0:1789760779.692785:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.692792:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.692808:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.726187:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973948 is committed 00002000:00010000:2.0:1789760779.726399:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27152384 -> 27156480 00010000:00010000:2.0:1789760779.726421:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.726438:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b6d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.726449:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.726457:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760779.726463:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.726474:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b6d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.726499:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b6d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.726508:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b6d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27152384->27156479] (req 27152384->27156479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.726698:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493833856:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34005us (34318us total) trans 4294973948 rc 0/0 00000100:00100000:2.0:1789760779.726714:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6659 00000100:00100000:2.0:1789760779.726727:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790001510513 time 790033707768 state 0x2 ref 0 00000100:00100000:2.0:1789760779.731126:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760779.734109:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493833984 00000100:00100000:3.0:1789760779.734132:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493833984 00000100:00100000:3.0:1789760779.734226:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790001510513 time 790033707768 state 0x0 ref 1 00000100:00100000:3.0:1789760779.734246:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790033968184 time 790033707768 state 0x1 ref 1 00000100:00100000:3.0:1789760779.734346:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790033968184 time 790075381408 state 0x0 ref 1 00000100:00100000:3.0:1789760779.734362:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c3094700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493833984:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760779.734432:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27156480, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd007 00010000:00010000:3.0:1789760779.734444:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b74 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.734465:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=790s v=(0 0) 00010000:00010000:3.0:1789760779.734469:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.734476:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.734493:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.780006:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973949 is committed 00002000:00010000:3.0:1789760779.780067:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27156480 -> 27160576 00010000:00010000:3.0:1789760779.780088:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.780105:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b74 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.780115:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.780123:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760779.780128:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.780139:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b74 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.780169:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b74 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.780176:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b74 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27156480->27160575] (req 27156480->27160575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.780366:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c3094700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493833984:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46000us (49317us total) trans 4294973949 rc 0/0 00000100:00100000:3.0:1789760779.780379:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6660 00000100:00100000:3.0:1789760779.780392:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790033968184 time 790075381408 state 0x2 ref 0 00000100:00100000:1.0:1789760779.788703:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760779.789719:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493834112 00000100:00100000:0.0:1789760779.789739:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493834112 00000100:00100000:0.0:1789760779.789828:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790033968184 time 790075381408 state 0x0 ref 1 00000100:00100000:0.0:1789760779.789847:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790075641824 time 790075381408 state 0x1 ref 1 00000100:00100000:0.0:1789760779.789889:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790075641824 time 790130924664 state 0x0 ref 1 00000100:00100000:0.0:1789760779.789904:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4363480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493834112:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760779.790017:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27160576, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd007 00010000:00010000:0.0:1789760779.790027:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06b7b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.790048:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=790s v=(0 0) 00010000:00010000:0.0:1789760779.790055:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.790063:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06b7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.790082:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06b7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.825928:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973950 is committed 00002000:00010000:2.0:1789760779.826219:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27160576 -> 27164672 00010000:00010000:2.0:1789760779.826240:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06b7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.826322:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06b7b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.826332:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06b7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.826339:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760779.826344:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06b7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.826354:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06b7b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.826381:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06b7b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.826389:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e06b7b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27160576->27164671] (req 27160576->27164671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.826568:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4363480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493834112:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36658us (37870us total) trans 4294973950 rc 0/0 00000100:00100000:2.0:1789760779.826582:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6661 00000100:00100000:2.0:1789760779.826594:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790075641824 time 790130924664 state 0x2 ref 0 00000100:00100000:1.0:1789760779.830347:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760779.830528:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493834240 00000100:00100000:0.0:1789760779.830541:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493834240 00000100:00100000:0.0:1789760779.830628:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790075641824 time 790130924664 state 0x0 ref 1 00000100:00100000:0.0:1789760779.830647:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790131185080 time 790130924664 state 0x1 ref 1 00000100:00100000:0.0:1789760779.830724:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790131185080 time 790171757541 state 0x0 ref 1 00000100:00100000:0.0:1789760779.830739:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493834240:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760779.830806:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27164672, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd007 00010000:00010000:0.0:1789760779.830817:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06b82 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.830836:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=790s v=(0 0) 00010000:00010000:0.0:1789760779.830842:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.830851:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06b82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.830873:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06b82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.839383:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760779.876388:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973951 is committed 00002000:00010000:2.0:1789760779.876464:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27164672 -> 27168768 00010000:00010000:2.0:1789760779.876486:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06b82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.876499:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06b82 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.876508:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06b82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.876516:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760779.876521:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06b82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.876531:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06b82 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.876556:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06b82 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.876570:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06b82 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27164672->27168767] (req 27164672->27168767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.876694:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493834240:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45951us (46355us total) trans 4294973951 rc 0/0 00000100:00100000:2.0:1789760779.876707:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6662 00000100:00100000:2.0:1789760779.876720:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790131185080 time 790171757541 state 0x2 ref 0 00000100:00100000:1.0:1789760779.885254:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760779.885697:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493834368 00000100:00100000:2.0:1789760779.885781:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493834368 00000100:00100000:2.0:1789760779.885859:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790131185080 time 790171757541 state 0x0 ref 1 00000100:00100000:2.0:1789760779.885879:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790172017957 time 790171757541 state 0x1 ref 1 00000100:00100000:2.0:1789760779.885927:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790172017957 time 790226961736 state 0x0 ref 1 00000100:00100000:2.0:1789760779.885942:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493834368:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760779.886049:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27168768, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd007 00010000:00010000:2.0:1789760779.886060:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b89 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.886080:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=790s v=(0 0) 00010000:00010000:2.0:1789760779.886085:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.886093:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.886114:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760779.916503:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27168768 -> 27172864 00010000:00010000:2.0:1789760779.916533:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.916550:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b89 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.916561:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.916570:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760779.916576:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.916587:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b89 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.916611:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b89 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.916619:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b89 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27168768->27172863] (req 27168768->27172863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.916754:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493834368:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30806us (31504us total) trans 4294973952 rc 0/0 00000100:00100000:2.0:1789760779.916767:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6663 00000100:00100000:2.0:1789760779.916779:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790172017957 time 790226961736 state 0x2 ref 0 00000001:00080000:2.0:1789760779.917259:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973952 is committed 00000100:00100000:0.0:1789760779.920797:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760779.920957:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493834496 00000100:00100000:3.0:1789760779.921001:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493834496 00000100:00100000:3.0:1789760779.921076:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790172017957 time 790226961736 state 0x0 ref 1 00000100:00100000:3.0:1789760779.921096:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790227222152 time 790226961736 state 0x1 ref 1 00000100:00100000:3.0:1789760779.921124:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790227222152 time 790262159586 state 0x0 ref 1 00000100:00100000:3.0:1789760779.921138:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493834496:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760779.921365:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27172864, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd007 00010000:00010000:3.0:1789760779.921377:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b90 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.921404:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=790s v=(0 0) 00010000:00010000:3.0:1789760779.921410:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.921418:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.921434:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760779.962454:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27172864 -> 27176960 00010000:00010000:0.0:1789760779.962481:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.962497:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b90 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.962507:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.962516:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760779.962521:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.962532:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b90 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.962556:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b90 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.962564:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06b90 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27172864->27176959] (req 27172864->27176959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.962685:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493834496:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41543us (41898us total) trans 4294973953 rc 0/0 00000100:00100000:0.0:1789760779.962698:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6664 00000100:00100000:0.0:1789760779.962710:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790227222152 time 790262159586 state 0x2 ref 0 00000001:00080000:2.0:1789760779.962810:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973953 is committed 00000100:00100000:0.0:1789760779.965557:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760779.965816:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493834624 00000100:00100000:2.0:1789760779.965829:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493834624 00000100:00100000:2.0:1789760779.965902:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790227222152 time 790262159586 state 0x0 ref 1 00000100:00100000:2.0:1789760779.965922:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790262420002 time 790262159586 state 0x1 ref 1 00000100:00100000:0.0:1789760779.965995:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790262420002 time 790307026920 state 0x0 ref 1 00000100:00100000:0.0:1789760779.966011:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493834624:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760779.966073:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27176960, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd007 00010000:00010000:0.0:1789760779.966082:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06b97 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.966098:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=790s v=(0 0) 00010000:00010000:0.0:1789760779.966102:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.966107:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06b97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.966123:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06b97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.999762:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973954 is committed 00002000:00010000:0.0:1789760780.000053:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27176960 -> 27181056 00010000:00010000:0.0:1789760780.000072:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06b97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.000088:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06b97 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.000098:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06b97 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.000106:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760780.000111:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06b97 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.000121:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06b97 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.000145:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06b97 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.000153:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06b97 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27176960->27181055] (req 27176960->27181055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.000295:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493834624:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34280us (34745us total) trans 4294973954 rc 0/0 00000100:00100000:0.0:1789760780.001239:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6665 00000100:00100000:0.0:1789760780.001254:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790262420002 time 790307026920 state 0x2 ref 0 00000100:00100000:3.0:1789760780.005626:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760780.006113:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493834752 00000100:00100000:2.0:1789760780.006125:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493834752 00000100:00100000:2.0:1789760780.006213:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790262420002 time 790307026920 state 0x0 ref 1 00000100:00100000:2.0:1789760780.006499:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790307287336 time 790307026920 state 0x1 ref 1 00000100:00100000:2.0:1789760780.006538:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790307287336 time 790347572730 state 0x0 ref 1 00000100:00100000:2.0:1789760780.006555:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493834752:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760780.006624:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27181056, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd007 00010000:00010000:2.0:1789760780.006636:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b9e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.006656:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=790s v=(0 0) 00010000:00010000:2.0:1789760780.006662:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.006737:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.006756:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.057282:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973955 is committed 00002000:00010000:0.0:1789760780.057557:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27181056 -> 27185152 00010000:00010000:0.0:1789760780.057578:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.057592:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b9e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.057602:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.057611:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760780.057617:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.057627:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b9e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.057653:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b9e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.057661:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06b9e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27181056->27185151] (req 27181056->27185151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.057817:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493834752:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51252us (52192us total) trans 4294973955 rc 0/0 00000100:00100000:0.0:1789760780.057831:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6666 00000100:00100000:0.0:1789760780.057844:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790307287336 time 790347572730 state 0x2 ref 0 00000100:00100000:1.0:1789760780.063607:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760780.063834:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493834880 00000100:00100000:2.0:1789760780.063848:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493834880 00000100:00100000:2.0:1789760780.063927:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790307287336 time 790347572730 state 0x0 ref 1 00000100:00100000:2.0:1789760780.063947:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790347833146 time 790347572730 state 0x1 ref 1 00000100:00100000:2.0:1789760780.064065:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790347833146 time 790405099368 state 0x0 ref 1 00000100:00100000:2.0:1789760780.064081:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc352c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493834880:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760780.064145:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27185152, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd007 00010000:00010000:2.0:1789760780.064157:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06ba5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.064178:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=790s v=(0 0) 00010000:00010000:2.0:1789760780.064183:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.064200:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06ba5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.064218:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06ba5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.107509:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973956 is committed 00002000:00010000:2.0:1789760780.107642:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27185152 -> 27189248 00010000:00010000:2.0:1789760780.107662:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06ba5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.107751:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06ba5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.107763:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06ba5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.107773:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760780.107779:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06ba5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.107863:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06ba5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.107889:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06ba5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.107899:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06ba5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27185152->27189247] (req 27185152->27189247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.108075:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493834880:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43990us (44474us total) trans 4294973956 rc 0/0 00000100:00100000:2.0:1789760780.108088:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6667 00000100:00100000:2.0:1789760780.108099:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790347833146 time 790405099368 state 0x2 ref 0 00000100:00100000:0.0:1789760780.111165:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760780.111268:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493835008 00000100:00100000:3.0:1789760780.111280:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493835008 00000100:00100000:3.0:1789760780.111347:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790347833146 time 790405099368 state 0x0 ref 1 00000100:00100000:3.0:1789760780.111364:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790405359784 time 790405099368 state 0x1 ref 1 00000100:00100000:3.0:1789760780.111394:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790405359784 time 790452429833 state 0x0 ref 1 00000100:00100000:3.0:1789760780.111407:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493835008:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760780.111468:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27189248, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd007 00010000:00010000:3.0:1789760780.111478:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06bac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.111496:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=790s v=(0 0) 00010000:00010000:3.0:1789760780.111501:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.111509:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06bac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.111525:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06bac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.148903:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973957 is committed 00002000:00010000:0.0:1789760780.149048:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27189248 -> 27193344 00010000:00010000:0.0:1789760780.149068:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06bac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.149082:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06bac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.149092:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06bac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.149102:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760780.149107:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06bac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.149117:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06bac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.149140:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06bac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.149155:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06bac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27189248->27193343] (req 27189248->27193343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.149292:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493835008:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37878us (38132us total) trans 4294973957 rc 0/0 00000100:00100000:0.0:1789760780.149304:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6668 00000100:00100000:0.0:1789760780.149316:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790405359784 time 790452429833 state 0x2 ref 0 00000100:00100000:1.0:1789760780.154102:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760780.154233:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493835136 00000100:00100000:0.0:1789760780.154255:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493835136 00000100:00100000:0.0:1789760780.154329:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790405359784 time 790452429833 state 0x0 ref 1 00000100:00100000:0.0:1789760780.154348:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790452690249 time 790452429833 state 0x1 ref 1 00000100:00100000:0.0:1789760780.154375:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790452690249 time 790495410311 state 0x0 ref 1 00000100:00100000:0.0:1789760780.154388:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493835136:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760780.154456:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27193344, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd007 00010000:00010000:0.0:1789760780.154467:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06bb3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.154487:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=790s v=(0 0) 00010000:00010000:0.0:1789760780.154493:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.154500:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06bb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.154523:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06bb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.155065:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620c7aa7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519260416:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760780.155137:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760780.155238:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519260416 00000100:00100000:1.0:1789760780.155253:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519260416 00000100:00100000:1.0:1789760780.155295:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 215 00000100:00100000:1.0:1789760780.155304:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519260416:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760780.155333:0:6690:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760780.155385:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519260416:12345-0@lo:13:osp-pre-1-0.0 Request processed in 81us (246us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760780.155394:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 215 00000100:00100000:1.0:1789760780.155455:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620c7aa7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519260416:0@lo:13:osp-pre-1-0.0 00000001:00080000:1.0:1789760780.213047:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973958 is committed 00002000:00010000:0.0:1789760780.213182:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27193344 -> 27197440 00010000:00010000:0.0:1789760780.213212:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06bb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.213227:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06bb3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.213237:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06bb3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.213253:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760780.213259:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06bb3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.213268:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06bb3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.213295:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06bb3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.213305:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06bb3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27193344->27197439] (req 27193344->27197439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.213452:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493835136:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59058us (59359us total) trans 4294973958 rc 0/0 00000100:00100000:0.0:1789760780.213464:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6669 00000100:00100000:0.0:1789760780.213475:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790452690249 time 790495410311 state 0x2 ref 0 00000100:00100000:1.0:1789760780.219947:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760780.220219:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493835264 00000100:00100000:0.0:1789760780.220234:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493835264 00000100:00100000:0.0:1789760780.220380:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790452690249 time 790495410311 state 0x0 ref 1 00000100:00100000:0.0:1789760780.220400:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790495670727 time 790495410311 state 0x1 ref 1 00000100:00100000:0.0:1789760780.220488:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790495670727 time 790561522721 state 0x0 ref 1 00000100:00100000:0.0:1789760780.220504:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493835264:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760780.220574:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27197440, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd007 00010000:00010000:0.0:1789760780.220585:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06bba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.220607:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=790s v=(0 0) 00010000:00010000:0.0:1789760780.220613:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.220622:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06bba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.220640:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06bba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760780.320354:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27197440 -> 27201536 00010000:00010000:3.0:1789760780.320384:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06bba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.320401:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06bba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.320411:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06bba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.320419:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760780.320425:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06bba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.320436:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06bba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.320461:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06bba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.320470:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06bba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27197440->27201535] (req 27197440->27201535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.320619:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493835264:12345-192.168.203.60@tcp:4:dd.0 Request processed in 100110us (100680us total) trans 4294973959 rc 0/0 00000100:00100000:3.0:1789760780.320633:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6670 00000100:00100000:3.0:1789760780.320645:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790495670727 time 790561522721 state 0x2 ref 0 00000001:00080000:2.0:1789760780.321431:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973959 is committed 00000100:00100000:1.0:1789760780.327682:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760780.328133:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493835392 00000100:00100000:3.0:1789760780.328148:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493835392 00000100:00100000:3.0:1789760780.328236:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790495670727 time 790561522721 state 0x0 ref 1 00000100:00100000:3.0:1789760780.328389:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790561783137 time 790561522721 state 0x1 ref 1 00000100:00100000:3.0:1789760780.328424:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790561783137 time 790669459420 state 0x0 ref 1 00000100:00100000:3.0:1789760780.328440:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493835392:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760780.328506:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27201536, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd007 00010000:00010000:3.0:1789760780.328517:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06bc1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.328536:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=790s v=(0 0) 00010000:00010000:3.0:1789760780.328542:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.328550:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06bc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.328566:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06bc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.373446:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973960 is committed 00002000:00010000:0.0:1789760780.373714:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27201536 -> 27205632 00010000:00010000:0.0:1789760780.373737:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06bc1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.373751:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06bc1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.373761:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06bc1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.373769:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760780.373776:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06bc1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.373786:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06bc1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.373813:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06bc1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.373821:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06bc1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27201536->27205631] (req 27201536->27205631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.374028:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493835392:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45583us (46354us total) trans 4294973960 rc 0/0 00000100:00100000:0.0:1789760780.374047:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6671 00000100:00100000:0.0:1789760780.374061:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790561783137 time 790669459420 state 0x2 ref 0 00000100:00100000:0.0:1789760780.383205:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760780.383593:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493835520 00000100:00100000:3.0:1789760780.383608:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493835520 00000100:00100000:2.0:1789760780.383852:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790561783137 time 790669459420 state 0x0 ref 1 00000100:00100000:2.0:1789760780.383871:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790669719836 time 790669459420 state 0x1 ref 1 00000100:00100000:2.0:1789760780.383944:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790669719836 time 790724978726 state 0x0 ref 1 00000100:00100000:2.0:1789760780.383987:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493835520:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760780.384065:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27205632, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd007 00010000:00010000:2.0:1789760780.384076:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06bc8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.384097:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=790s v=(0 0) 00010000:00010000:2.0:1789760780.384102:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.384110:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06bc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.384128:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06bc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.433014:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973961 is committed 00002000:00010000:2.0:1789760780.433158:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27205632 -> 27209728 00010000:00010000:2.0:1789760780.433178:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06bc8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.433204:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06bc8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.433214:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06bc8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.433222:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760780.433228:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06bc8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.433248:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06bc8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.433273:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06bc8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.433280:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06bc8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27205632->27209727] (req 27205632->27209727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.433409:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493835520:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49445us (50223us total) trans 4294973961 rc 0/0 00000100:00100000:2.0:1789760780.433426:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6672 00000100:00100000:2.0:1789760780.433439:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790669719836 time 790724978726 state 0x2 ref 0 00000100:00100000:0.0:1789760780.436567:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760780.436709:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493835648 00000100:00100000:2.0:1789760780.436723:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493835648 00000100:00100000:2.0:1789760780.436798:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790669719836 time 790724978726 state 0x0 ref 1 00000100:00100000:2.0:1789760780.436817:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790725239142 time 790724978726 state 0x1 ref 1 00000100:00100000:2.0:1789760780.436842:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790725239142 time 790777877491 state 0x0 ref 1 00000100:00100000:2.0:1789760780.436857:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493835648:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760780.436931:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27209728, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd007 00010000:00010000:2.0:1789760780.436942:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06bcf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.436991:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=790s v=(0 0) 00010000:00010000:2.0:1789760780.436997:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.437006:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06bcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.437028:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06bcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.480818:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973962 is committed 00002000:00010000:3.0:1789760780.480993:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27209728 -> 27213824 00010000:00010000:3.0:1789760780.481015:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06bcf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.481030:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06bcf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.481039:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06bcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.481048:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760780.481054:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06bcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.481064:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06bcf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.481092:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06bcf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.481101:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06bcf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27209728->27213823] (req 27209728->27213823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.481382:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493835648:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44519us (44821us total) trans 4294973962 rc 0/0 00000100:00100000:3.0:1789760780.481396:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6673 00000100:00100000:3.0:1789760780.481408:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790725239142 time 790777877491 state 0x2 ref 0 00000100:00100000:0.0:1789760780.486380:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760780.486538:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493835776 00000100:00100000:3.0:1789760780.486553:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493835776 00000100:00100000:3.0:1789760780.486625:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790725239142 time 790777877491 state 0x0 ref 1 00000100:00100000:3.0:1789760780.486644:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790778137907 time 790777877491 state 0x1 ref 1 00000100:00100000:3.0:1789760780.486741:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790778137907 time 790827774727 state 0x0 ref 1 00000100:00100000:3.0:1789760780.486756:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493835776:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760780.486827:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27213824, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd007 00010000:00010000:3.0:1789760780.486837:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06bd6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.486857:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=790s v=(0 0) 00010000:00010000:3.0:1789760780.486862:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.486869:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06bd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.486884:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06bd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.522640:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973963 is committed 00002000:00010000:3.0:1789760780.523236:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27213824 -> 27217920 00010000:00010000:3.0:1789760780.523313:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06bd6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.523329:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06bd6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.523340:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06bd6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.523348:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760780.523354:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06bd6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.523366:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06bd6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.523396:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06bd6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.523405:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06bd6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27213824->27217919] (req 27213824->27217919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.523630:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493835776:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36869us (37258us total) trans 4294973963 rc 0/0 00000100:00100000:3.0:1789760780.523645:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6674 00000100:00100000:3.0:1789760780.523657:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790778137907 time 790827774727 state 0x2 ref 0 00000100:00100000:0.0:1789760780.527404:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760780.527508:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493835904 00000100:00100000:3.0:1789760780.527522:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493835904 00000100:00100000:3.0:1789760780.527595:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790778137907 time 790827774727 state 0x0 ref 1 00000100:00100000:3.0:1789760780.527615:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790828035143 time 790827774727 state 0x1 ref 1 00000100:00100000:3.0:1789760780.527693:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790828035143 time 790868727683 state 0x0 ref 1 00000100:00100000:3.0:1789760780.527709:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493835904:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760780.527776:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27217920, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd007 00010000:00010000:3.0:1789760780.527787:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06bdd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.527806:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=790s v=(0 0) 00010000:00010000:3.0:1789760780.527812:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.527819:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06bdd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.527835:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06bdd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.560677:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973964 is committed 00002000:00010000:1.0:1789760780.560858:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27217920 -> 27222016 00010000:00010000:1.0:1789760780.560874:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06bdd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.560886:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06bdd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.560895:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06bdd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.560902:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760780.560907:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06bdd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.560922:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06bdd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.560941:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06bdd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.560949:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06bdd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27217920->27222015] (req 27217920->27222015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.561055:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493835904:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33345us (33662us total) trans 4294973964 rc 0/0 00000100:00100000:1.0:1789760780.561066:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6675 00000100:00100000:1.0:1789760780.561076:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790828035143 time 790868727683 state 0x2 ref 0 00000100:00100000:0.0:1789760780.564661:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760780.564875:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493836032 00000100:00100000:2.0:1789760780.564900:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493836032 00000100:00100000:2.0:1789760780.565008:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790828035143 time 790868727683 state 0x0 ref 1 00000100:00100000:2.0:1789760780.565028:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790868988099 time 790868727683 state 0x1 ref 1 00000100:00100000:2.0:1789760780.565118:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790868988099 time 790906151902 state 0x0 ref 1 00000100:00100000:2.0:1789760780.565133:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493836032:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760780.565210:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27222016, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd007 00010000:00010000:2.0:1789760780.565222:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06be4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.565242:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=790s v=(0 0) 00010000:00010000:2.0:1789760780.565247:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.565256:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06be4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.565273:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06be4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.591370:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973965 is committed 00002000:00010000:0.0:1789760780.591497:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27222016 -> 27226112 00010000:00010000:0.0:1789760780.591517:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06be4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.591542:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06be4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.591553:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06be4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.591561:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760780.591567:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06be4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.591577:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06be4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.591606:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06be4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.591614:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06be4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27222016->27226111] (req 27222016->27226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.591737:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493836032:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26599us (27084us total) trans 4294973965 rc 0/0 00000100:00100000:0.0:1789760780.591750:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6676 00000100:00100000:0.0:1789760780.591763:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790868988099 time 790906151902 state 0x2 ref 0 00000100:00100000:0.0:1789760780.598080:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760780.598295:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493836160 00000100:00100000:3.0:1789760780.598311:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493836160 00000100:00100000:3.0:1789760780.598390:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790868988099 time 790906151902 state 0x0 ref 1 00000100:00100000:3.0:1789760780.598412:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790906412318 time 790906151902 state 0x1 ref 1 00000100:00100000:3.0:1789760780.598446:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790906412318 time 790939481060 state 0x0 ref 1 00000100:00100000:3.0:1789760780.598465:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493836160:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760780.598534:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27226112, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xd007 00010000:00010000:3.0:1789760780.598544:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06beb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.598566:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=790s v=(0 0) 00010000:00010000:3.0:1789760780.598576:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.598585:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06beb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.598603:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06beb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.631771:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973966 is committed 00002000:00010000:3.0:1789760780.631895:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27226112 -> 27230208 00002000:00010000:3.0:1789760780.631900:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760779 -> 1789760780 00002000:00010000:3.0:1789760780.631903:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760779 -> 1789760780 00010000:00010000:3.0:1789760780.631920:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06beb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.631936:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06beb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.631945:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06beb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.631953:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760780.631990:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06beb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.632002:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06beb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.632027:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06beb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.632034:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06beb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27226112->27230207] (req 27226112->27230207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.632150:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493836160:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33680us (34078us total) trans 4294973966 rc 0/0 00000100:00100000:3.0:1789760780.632162:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6677 00000100:00100000:3.0:1789760780.632174:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790906412318 time 790939481060 state 0x2 ref 0 00000100:00100000:1.0:1789760780.634935:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760780.635076:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493836288 00000100:00100000:3.0:1789760780.635087:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493836288 00000100:00100000:3.0:1789760780.635155:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790906412318 time 790939481060 state 0x0 ref 1 00000100:00100000:3.0:1789760780.635173:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790939741476 time 790939481060 state 0x1 ref 1 00000100:00100000:3.0:1789760780.635360:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790939741476 time 790976394588 state 0x0 ref 1 00000100:00100000:3.0:1789760780.635375:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493836288:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760780.635444:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27230208, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd007 00010000:00010000:3.0:1789760780.635456:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06bf2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.635475:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=790s v=(0 0) 00010000:00010000:3.0:1789760780.635480:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.635487:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06bf2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.635505:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06bf2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760780.677384:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27230208 -> 27234304 00010000:00010000:0.0:1789760780.677408:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06bf2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.677423:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06bf2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.677433:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06bf2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.677442:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760780.677447:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06bf2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.677460:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06bf2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.677460:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973967 is committed 00010000:00010000:0.0:1789760780.677496:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06bf2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.677507:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06bf2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27230208->27234303] (req 27230208->27234303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.677604:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493836288:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42227us (42677us total) trans 4294973967 rc 0/0 00000100:00100000:0.0:1789760780.677616:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6678 00000100:00100000:0.0:1789760780.677628:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790939741476 time 790976394588 state 0x2 ref 0 00000100:00100000:0.0:1789760780.686069:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760780.686169:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493836416 00000100:00100000:1.0:1789760780.686181:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493836416 00000100:00100000:1.0:1789760780.686245:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790939741476 time 790976394588 state 0x0 ref 1 00000100:00100000:1.0:1789760780.686261:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790976655004 time 790976394588 state 0x1 ref 1 00000100:00100000:1.0:1789760780.686293:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790976655004 time 791027328205 state 0x0 ref 1 00000100:00100000:1.0:1789760780.686306:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8794700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493836416:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760780.686353:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27234304, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd007 00010000:00010000:1.0:1789760780.686363:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06bf9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.686379:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=791s v=(0 0) 00010000:00010000:1.0:1789760780.686384:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.686390:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06bf9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.686406:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06bf9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.739600:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973968 is committed 00002000:00010000:3.0:1789760780.741560:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27234304 -> 27238400 00010000:00010000:3.0:1789760780.741588:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06bf9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.741605:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06bf9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.741615:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06bf9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.741625:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760780.741630:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06bf9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.741642:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06bf9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.741674:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06bf9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.741684:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06bf9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27234304->27238399] (req 27234304->27238399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.741824:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8794700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493836416:12345-192.168.203.60@tcp:4:dd.0 Request processed in 55512us (55757us total) trans 4294973968 rc 0/0 00000100:00100000:3.0:1789760780.741836:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6679 00000100:00100000:3.0:1789760780.741858:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790976655004 time 791027328205 state 0x2 ref 0 00000100:00100000:2.0:1789760780.746363:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760780.746690:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493836544 00000100:00100000:0.0:1789760780.746711:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493836544 00000100:00100000:0.0:1789760780.746787:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 790976655004 time 791027328205 state 0x0 ref 1 00000100:00100000:0.0:1789760780.746807:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791027588621 time 791027328205 state 0x1 ref 1 00000100:00100000:0.0:1789760780.746845:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791027588621 time 791087879965 state 0x0 ref 1 00000100:00100000:0.0:1789760780.746860:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3663480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493836544:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760780.746930:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27238400, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd007 00010000:00010000:0.0:1789760780.746941:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06c00 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.747000:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=791s v=(0 0) 00010000:00010000:0.0:1789760780.747006:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.747014:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06c00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.747039:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06c00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.787246:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973969 is committed 00002000:00010000:0.0:1789760780.787581:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27238400 -> 27242496 00010000:00010000:0.0:1789760780.787601:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06c00 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.787617:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06c00 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.787628:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06c00 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.787636:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760780.787641:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06c00 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.787652:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06c00 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.787678:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06c00 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.787686:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06c00 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27238400->27242495] (req 27238400->27242495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.787811:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3663480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493836544:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40947us (41458us total) trans 4294973969 rc 0/0 00000100:00100000:0.0:1789760780.787824:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6680 00000100:00100000:0.0:1789760780.787837:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791027588621 time 791087879965 state 0x2 ref 0 00000100:00100000:1.0:1789760780.792425:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760780.792765:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493836672 00000100:00100000:0.0:1789760780.792780:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493836672 00000100:00100000:0.0:1789760780.792860:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791027588621 time 791087879965 state 0x0 ref 1 00000100:00100000:0.0:1789760780.792880:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791088140381 time 791087879965 state 0x1 ref 1 00000100:00100000:3.0:1789760780.793546:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791088140381 time 791134577015 state 0x0 ref 1 00000100:00100000:3.0:1789760780.793568:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493836672:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760780.793638:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27242496, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd007 00010000:00010000:3.0:1789760780.793650:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06c07 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.793739:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=791s v=(0 0) 00010000:00010000:3.0:1789760780.793746:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.793753:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06c07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.793772:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06c07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.832543:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973970 is committed 00002000:00010000:1.0:1789760780.832674:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27242496 -> 27246592 00010000:00010000:1.0:1789760780.832694:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06c07 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.832711:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06c07 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.832722:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06c07 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.832729:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760780.832736:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06c07 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.832747:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06c07 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.832774:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06c07 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.832783:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06c07 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27242496->27246591] (req 27242496->27246591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.832907:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493836672:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39340us (40489us total) trans 4294973970 rc 0/0 00000100:00100000:1.0:1789760780.832919:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6681 00000100:00100000:1.0:1789760780.832933:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791088140381 time 791134577015 state 0x2 ref 0 00000100:00100000:2.0:1789760780.837715:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760780.837836:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493836800 00000100:00100000:3.0:1789760780.837850:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493836800 00000100:00100000:3.0:1789760780.837923:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791088140381 time 791134577015 state 0x0 ref 1 00000100:00100000:3.0:1789760780.837942:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791134837431 time 791134577015 state 0x1 ref 1 00000100:00100000:3.0:1789760780.837997:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791134837431 time 791179031968 state 0x0 ref 1 00000100:00100000:3.0:1789760780.838012:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3662d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493836800:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760780.838082:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27246592, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd007 00010000:00010000:3.0:1789760780.838092:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06c0e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.838110:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=791s v=(0 0) 00010000:00010000:3.0:1789760780.838115:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.838122:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06c0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.838143:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06c0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.859389:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760780.881193:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973971 is committed 00002000:00010000:3.0:1789760780.881407:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27246592 -> 27250688 00010000:00010000:3.0:1789760780.881429:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06c0e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.881448:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06c0e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.881456:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06c0e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.881464:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760780.881469:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06c0e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.881478:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06c0e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.881502:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06c0e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.881510:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06c0e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27246592->27250687] (req 27246592->27250687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.881687:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3662d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493836800:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43668us (43977us total) trans 4294973971 rc 0/0 00000100:00100000:3.0:1789760780.881702:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6682 00000100:00100000:3.0:1789760780.881715:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791134837431 time 791179031968 state 0x2 ref 0 00000100:00100000:2.0:1789760780.885504:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760780.885615:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493836928 00000100:00100000:0.0:1789760780.885687:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493836928 00000100:00100000:0.0:1789760780.886079:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791134837431 time 791179031968 state 0x0 ref 1 00000100:00100000:0.0:1789760780.886100:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791179292384 time 791179031968 state 0x1 ref 1 00000100:00100000:0.0:1789760780.886129:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791179292384 time 791227165250 state 0x0 ref 1 00000100:00100000:0.0:1789760780.886145:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3662680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493836928:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760780.886239:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27250688, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd007 00010000:00010000:0.0:1789760780.886250:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06c15 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.886271:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=791s v=(0 0) 00010000:00010000:0.0:1789760780.886276:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.886284:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06c15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.886301:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06c15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760780.924541:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27250688 -> 27254784 00010000:00010000:0.0:1789760780.924568:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06c15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.924584:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06c15 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.924593:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06c15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.924603:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760780.924608:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06c15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.924619:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06c15 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.924645:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06c15 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.924653:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06c15 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27250688->27254783] (req 27250688->27254783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.924784:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3662680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493836928:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38636us (39293us total) trans 4294973972 rc 0/0 00000100:00100000:0.0:1789760780.924797:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6683 00000100:00100000:0.0:1789760780.924810:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791179292384 time 791227165250 state 0x2 ref 0 00000001:00080000:2.0:1789760780.924995:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973972 is committed 00000100:00100000:1.0:1789760780.928229:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760780.928324:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493837056 00000100:00100000:0.0:1789760780.928337:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493837056 00000100:00100000:0.0:1789760780.928408:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791179292384 time 791227165250 state 0x0 ref 1 00000100:00100000:0.0:1789760780.928427:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791227425666 time 791227165250 state 0x1 ref 1 00000100:00100000:0.0:1789760780.928460:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791227425666 time 791269495868 state 0x0 ref 1 00000100:00100000:0.0:1789760780.928474:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493837056:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760780.928539:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27254784, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd007 00010000:00010000:0.0:1789760780.928550:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06c1c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.928568:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=791s v=(0 0) 00010000:00010000:0.0:1789760780.928575:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.928584:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06c1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.928601:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06c1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.969673:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973973 is committed 00002000:00010000:1.0:1789760780.969919:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27254784 -> 27258880 00010000:00010000:1.0:1789760780.969941:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06c1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.969957:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06c1c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.970006:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06c1c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.970015:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760780.970020:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06c1c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.970032:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06c1c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.970058:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06c1c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.970066:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06c1c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27254784->27258879] (req 27254784->27258879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.970311:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493837056:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41830us (42088us total) trans 4294973973 rc 0/0 00000100:00100000:1.0:1789760780.970324:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6684 00000100:00100000:1.0:1789760780.970337:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791227425666 time 791269495868 state 0x2 ref 0 00000100:00100000:3.0:1789760780.973143:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760780.973344:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493837184 00000100:00100000:1.0:1789760780.973356:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493837184 00000100:00100000:1.0:1789760780.973431:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791227425666 time 791269495868 state 0x0 ref 1 00000100:00100000:1.0:1789760780.973451:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791269756284 time 791269495868 state 0x1 ref 1 00000100:00100000:1.0:1789760780.973490:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791269756284 time 791314524450 state 0x0 ref 1 00000100:00100000:1.0:1789760780.973507:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493837184:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760780.973573:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27258880, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd007 00010000:00010000:1.0:1789760780.973584:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c23 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.973605:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=791s v=(0 0) 00010000:00010000:1.0:1789760780.973610:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.973618:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.973635:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.006326:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973974 is committed 00002000:00010000:1.0:1789760781.006341:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27258880 -> 27262976 00010000:00010000:1.0:1789760781.006513:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.006532:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c23 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.006543:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c23 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.006552:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760781.006558:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c23 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.006568:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c23 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.006595:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c23 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.006604:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c23 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27258880->27262975] (req 27258880->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.006843:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493837184:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33332us (33704us total) trans 4294973974 rc 0/0 00000100:00100000:1.0:1789760781.006857:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6685 00000100:00100000:1.0:1789760781.006869:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791269756284 time 791314524450 state 0x2 ref 0 00000100:00100000:0.0:1789760781.012553:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760781.012945:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493837312 00000100:00100000:3.0:1789760781.012994:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493837312 00000100:00080000:2.0:1789760781.013077:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760781.013110:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791269756284 time 791314524450 state 0x0 ref 1 00000100:00100000:3.0:1789760781.013131:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791314784866 time 791314524450 state 0x1 ref 1 00000100:00100000:3.0:1789760781.013168:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791314784866 time 791354203162 state 0x0 ref 1 00000100:00100000:3.0:1789760781.013192:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8796a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493837312:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760781.013259:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27262976, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd007 00010000:00010000:3.0:1789760781.013270:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06c2a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.013288:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=791s v=(0 0) 00010000:00010000:3.0:1789760781.013294:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.013301:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06c2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.013317:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06c2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.081358:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973975 is committed 00002000:00010000:1.0:1789760781.081532:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27262976 -> 27267072 00002000:00010000:1.0:1789760781.081539:0:6692:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb blocks from disk: 53255 -> 55305 00010000:00010000:1.0:1789760781.081556:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06c2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.081570:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06c2a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.081579:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06c2a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.081587:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760781.081593:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06c2a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.081604:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06c2a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.081634:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06c2a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.081642:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06c2a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->27267071] (req 27262976->27267071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.081904:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8796a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493837312:12345-192.168.203.60@tcp:4:dd.0 Request processed in 68715us (69357us total) trans 4294973975 rc 0/0 00000100:00100000:1.0:1789760781.081917:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6686 00000100:00100000:1.0:1789760781.081930:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791314784866 time 791354203162 state 0x2 ref 0 00000100:00100000:0.0:1789760781.089570:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760781.089896:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493837440 00000100:00100000:3.0:1789760781.089914:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493837440 00000100:00100000:3.0:1789760781.090084:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791314784866 time 791354203162 state 0x0 ref 1 00000100:00100000:3.0:1789760781.090104:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791354463578 time 791354203162 state 0x1 ref 1 00000100:00100000:3.0:1789760781.090142:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791354463578 time 791431176838 state 0x0 ref 1 00000100:00100000:3.0:1789760781.090156:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493837440:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760781.090301:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27267072, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd809 00010000:00010000:3.0:1789760781.090313:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c31 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.090333:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=791s v=(0 0) 00010000:00010000:3.0:1789760781.090338:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.090345:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.090362:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.133942:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973976 is committed 00002000:00010000:1.0:1789760781.134140:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27267072 -> 27271168 00010000:00010000:1.0:1789760781.134160:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.134176:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c31 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.134198:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c31 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.134206:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760781.134212:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c31 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.134227:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c31 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.134315:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c31 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.134324:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c31 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27267072->27271167] (req 27267072->27271167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.134448:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493837440:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44286us (44884us total) trans 4294973976 rc 0/0 00000100:00100000:1.0:1789760781.134460:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6687 00000100:00100000:1.0:1789760781.134473:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791354463578 time 791431176838 state 0x2 ref 0 00000100:00100000:2.0:1789760781.140437:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760781.140531:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493837568 00000100:00100000:1.0:1789760781.140545:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493837568 00000100:00100000:1.0:1789760781.140621:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791354463578 time 791431176838 state 0x0 ref 1 00000100:00100000:1.0:1789760781.140640:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791431437254 time 791431176838 state 0x1 ref 1 00000100:00100000:1.0:1789760781.140885:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791431437254 time 791481917850 state 0x0 ref 1 00000100:00100000:1.0:1789760781.140902:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3662300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493837568:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760781.141015:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27271168, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd809 00010000:00010000:1.0:1789760781.141027:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c38 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.141048:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=791s v=(0 0) 00010000:00010000:1.0:1789760781.141055:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.141063:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.141080:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.202087:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973977 is committed 00002000:00010000:1.0:1789760781.202323:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27271168 -> 27275264 00010000:00010000:1.0:1789760781.202345:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.202362:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c38 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.202373:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.202381:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760781.202386:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.202397:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c38 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.202422:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c38 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.202432:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c38 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27271168->27275263] (req 27271168->27275263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.202606:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3662300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493837568:12345-192.168.203.60@tcp:4:dd.0 Request processed in 61699us (62174us total) trans 4294973977 rc 0/0 00000100:00100000:1.0:1789760781.202619:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6688 00000100:00100000:1.0:1789760781.202633:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791431437254 time 791481917850 state 0x2 ref 0 00000100:00100000:0.0:1789760781.205885:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760781.206000:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493837696 00000100:00100000:1.0:1789760781.206014:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493837696 00000100:00100000:1.0:1789760781.206090:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791431437254 time 791481917850 state 0x0 ref 1 00000100:00100000:1.0:1789760781.206110:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791482178266 time 791481917850 state 0x1 ref 1 00000100:00100000:1.0:1789760781.206190:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791482178266 time 791547215334 state 0x0 ref 1 00000100:00100000:1.0:1789760781.206207:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493837696:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760781.206283:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27275264, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd809 00010000:00010000:1.0:1789760781.206293:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06c3f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.206314:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=791s v=(0 0) 00010000:00010000:1.0:1789760781.206319:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.206326:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06c3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.206349:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06c3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.245472:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973978 is committed 00002000:00010000:1.0:1789760781.245614:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27275264 -> 27279360 00010000:00010000:1.0:1789760781.245636:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06c3f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.245652:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06c3f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.245663:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06c3f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.245672:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760781.245677:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06c3f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.245744:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06c3f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.245773:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06c3f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.245782:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06c3f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27275264->27279359] (req 27275264->27279359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.245955:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493837696:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39744us (40076us total) trans 4294973978 rc 0/0 00000100:00100000:1.0:1789760781.246024:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6689 00000100:00100000:1.0:1789760781.246036:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791482178266 time 791547215334 state 0x2 ref 0 00000100:00100000:0.0:1789760781.250105:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760781.250401:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493837824 00000100:00100000:3.0:1789760781.250416:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493837824 00000100:00100000:3.0:1789760781.250494:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791482178266 time 791547215334 state 0x0 ref 1 00000100:00100000:3.0:1789760781.250513:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791547475750 time 791547215334 state 0x1 ref 1 00000100:00100000:3.0:1789760781.250548:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791547475750 time 791591582732 state 0x0 ref 1 00000100:00100000:3.0:1789760781.250563:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493837824:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760781.250637:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27279360, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd809 00010000:00010000:3.0:1789760781.250651:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06c46 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.250683:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=791s v=(0 0) 00010000:00010000:3.0:1789760781.250833:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.250846:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06c46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.250866:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06c46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.293664:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973979 is committed 00002000:00010000:1.0:1789760781.293846:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27279360 -> 27283456 00010000:00010000:1.0:1789760781.293868:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06c46 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.293885:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06c46 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.293896:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06c46 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.293905:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760781.293910:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06c46 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.293920:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06c46 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.293949:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06c46 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.293988:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06c46 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27279360->27283455] (req 27279360->27283455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.294198:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493837824:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43619us (44091us total) trans 4294973979 rc 0/0 00000100:00100000:1.0:1789760781.294211:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6690 00000100:00100000:1.0:1789760781.294224:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791547475750 time 791591582732 state 0x2 ref 0 00000100:00100000:2.0:1789760781.302370:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760781.302499:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493837952 00000100:00100000:3.0:1789760781.302515:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493837952 00000100:00100000:3.0:1789760781.302588:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791547475750 time 791591582732 state 0x0 ref 1 00000100:00100000:3.0:1789760781.302607:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791591843148 time 791591582732 state 0x1 ref 1 00000100:00100000:3.0:1789760781.302640:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791591843148 time 791643676140 state 0x0 ref 1 00000100:00100000:3.0:1789760781.302654:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3661500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493837952:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760781.302718:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27283456, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd809 00010000:00010000:3.0:1789760781.302728:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c4d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.302747:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=791s v=(0 0) 00010000:00010000:3.0:1789760781.302753:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.302760:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.302778:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.337026:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973980 is committed 00002000:00010000:1.0:1789760781.337151:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27283456 -> 27287552 00010000:00010000:1.0:1789760781.337171:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.337196:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c4d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.337207:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c4d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.337215:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760781.337221:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c4d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.337299:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c4d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.337333:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c4d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.337343:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c4d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27283456->27287551] (req 27283456->27287551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.337503:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3661500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493837952:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34843us (35143us total) trans 4294973980 rc 0/0 00000100:00100000:1.0:1789760781.337517:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6691 00000100:00100000:1.0:1789760781.337530:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791591843148 time 791643676140 state 0x2 ref 0 00000100:00100000:2.0:1789760781.340781:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760781.340885:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493838080 00000100:00100000:1.0:1789760781.340898:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493838080 00000100:00100000:1.0:1789760781.341006:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791591843148 time 791643676140 state 0x0 ref 1 00000100:00100000:1.0:1789760781.341026:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791643936556 time 791643676140 state 0x1 ref 1 00000100:00100000:1.0:1789760781.341057:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791643936556 time 791682092396 state 0x0 ref 1 00000100:00100000:1.0:1789760781.341073:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb452a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493838080:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760781.341137:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27287552, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd809 00010000:00010000:1.0:1789760781.341148:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06c54 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.341166:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=791s v=(0 0) 00010000:00010000:1.0:1789760781.341172:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.341189:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06c54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.341206:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06c54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.380027:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973981 is committed 00002000:00010000:1.0:1789760781.380150:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27287552 -> 27291648 00010000:00010000:1.0:1789760781.380170:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06c54 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.380197:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06c54 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.380207:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06c54 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.380216:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760781.380222:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06c54 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.380306:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06c54 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.380332:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06c54 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.380341:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06c54 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27287552->27291647] (req 27287552->27291647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.380509:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb452a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493838080:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39432us (39735us total) trans 4294973981 rc 0/0 00000100:00100000:1.0:1789760781.380532:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6692 00000100:00100000:1.0:1789760781.380544:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791643936556 time 791682092396 state 0x2 ref 0 00000100:00100000:0.0:1789760781.384649:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760781.384780:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493838208 00000100:00100000:1.0:1789760781.384794:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493838208 00000100:00100000:1.0:1789760781.384872:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791643936556 time 791682092396 state 0x0 ref 1 00000100:00100000:1.0:1789760781.384891:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791682352812 time 791682092396 state 0x1 ref 1 00000100:00100000:1.0:1789760781.384930:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791682352812 time 791725964991 state 0x0 ref 1 00000100:00100000:1.0:1789760781.384944:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493838208:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760781.385106:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27291648, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd809 00010000:00010000:1.0:1789760781.385117:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c5b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.385137:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=791s v=(0 0) 00010000:00010000:1.0:1789760781.385143:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.385151:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.385166:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.425916:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973982 is committed 00002000:00010000:1.0:1789760781.426072:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27291648 -> 27295744 00010000:00010000:1.0:1789760781.426092:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.426107:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c5b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.426117:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c5b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.426126:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760781.426131:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c5b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.426140:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c5b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.426165:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c5b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.426174:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c5b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27291648->27295743] (req 27291648->27295743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.426305:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493838208:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41355us (41667us total) trans 4294973982 rc 0/0 00000100:00100000:1.0:1789760781.426321:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6693 00000100:00100000:1.0:1789760781.426332:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791682352812 time 791725964991 state 0x2 ref 0 00000100:00100000:2.0:1789760781.430066:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760781.430296:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493838336 00000100:00100000:3.0:1789760781.430310:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493838336 00000100:00100000:3.0:1789760781.430385:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791682352812 time 791725964991 state 0x0 ref 1 00000100:00100000:3.0:1789760781.430404:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791726225407 time 791725964991 state 0x1 ref 1 00000100:00100000:3.0:1789760781.430437:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791726225407 time 791771471873 state 0x0 ref 1 00000100:00100000:3.0:1789760781.430452:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb453800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493838336:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760781.430521:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27295744, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd809 00010000:00010000:3.0:1789760781.430531:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c62 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.430551:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=791s v=(0 0) 00010000:00010000:3.0:1789760781.430556:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.430564:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.430580:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.462763:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973983 is committed 00002000:00010000:3.0:1789760781.462773:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27295744 -> 27299840 00010000:00010000:3.0:1789760781.462795:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.462811:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c62 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.462822:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c62 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.462831:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760781.462838:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c62 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.462848:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c62 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.462873:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c62 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.462884:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c62 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27295744->27299839] (req 27295744->27299839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.463090:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb453800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493838336:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32633us (33030us total) trans 4294973983 rc 0/0 00000100:00100000:3.0:1789760781.463103:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6694 00000100:00100000:3.0:1789760781.463116:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791726225407 time 791771471873 state 0x2 ref 0 00000100:00100000:2.0:1789760781.470648:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760781.470768:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493838464 00000100:00100000:3.0:1789760781.470782:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493838464 00000100:00100000:3.0:1789760781.470860:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791726225407 time 791771471873 state 0x0 ref 1 00000100:00100000:3.0:1789760781.470880:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791771732289 time 791771471873 state 0x1 ref 1 00000100:00100000:3.0:1789760781.471070:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791771732289 time 791812103126 state 0x0 ref 1 00000100:00100000:3.0:1789760781.471086:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb17850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493838464:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760781.471155:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27299840, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd809 00010000:00010000:3.0:1789760781.471165:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c69 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.471195:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=791s v=(0 0) 00010000:00010000:3.0:1789760781.471201:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.471208:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.471226:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.504434:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973984 is committed 00002000:00010000:2.0:1789760781.504761:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27299840 -> 27303936 00010000:00010000:2.0:1789760781.504781:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.504796:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c69 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.504807:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.504816:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760781.504822:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.504838:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c69 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.504864:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c69 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.504872:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06c69 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27299840->27303935] (req 27299840->27303935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.505467:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb17850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493838464:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34374us (34824us total) trans 4294973984 rc 0/0 00000100:00100000:3.0:1789760781.505482:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6695 00000100:00100000:3.0:1789760781.505493:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791771732289 time 791812103126 state 0x2 ref 0 00000100:00100000:0.0:1789760781.515252:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760781.515372:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493838592 00000100:00100000:1.0:1789760781.515386:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493838592 00000100:00100000:1.0:1789760781.515465:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791771732289 time 791812103126 state 0x0 ref 1 00000100:00100000:1.0:1789760781.515486:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791812363542 time 791812103126 state 0x1 ref 1 00000100:00100000:1.0:1789760781.515525:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791812363542 time 791856559962 state 0x0 ref 1 00000100:00100000:1.0:1789760781.515540:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493838592:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760781.515610:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27303936, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd809 00010000:00010000:1.0:1789760781.515623:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c70 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.515653:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=791s v=(0 0) 00010000:00010000:1.0:1789760781.515666:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.515674:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.515691:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.561571:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973985 is committed 00002000:00010000:1.0:1789760781.561735:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27303936 -> 27308032 00010000:00010000:1.0:1789760781.561756:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.561771:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c70 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.561779:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c70 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.561786:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760781.561793:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c70 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.561803:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c70 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.561826:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c70 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.561833:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06c70 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27303936->27308031] (req 27303936->27308031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.562184:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493838592:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46577us (46890us total) trans 4294973985 rc 0/0 00000100:00100000:1.0:1789760781.562303:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6696 00000100:00100000:1.0:1789760781.562317:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791812363542 time 791856559962 state 0x2 ref 0 00000100:00100000:2.0:1789760781.567270:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760781.567627:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493838720 00000100:00100000:1.0:1789760781.567641:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493838720 00000100:00100000:0.0:1789760781.569929:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791812363542 time 791856559962 state 0x0 ref 1 00000100:00100000:0.0:1789760781.569954:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791856820378 time 791856559962 state 0x1 ref 1 00000100:00100000:0.0:1789760781.570062:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791856820378 time 791911096551 state 0x0 ref 1 00000100:00100000:0.0:1789760781.570086:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb452a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493838720:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760781.570153:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27308032, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd809 00010000:00010000:0.0:1789760781.570164:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06c77 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.570314:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=791s v=(0 0) 00010000:00010000:0.0:1789760781.570320:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.570328:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06c77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.570344:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06c77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.602731:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973986 is committed 00002000:00010000:1.0:1789760781.602853:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27308032 -> 27312128 00010000:00010000:1.0:1789760781.602874:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06c77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.602889:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06c77 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.602899:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06c77 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.602907:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760781.602913:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06c77 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.602923:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06c77 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.602950:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06c77 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.602981:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06c77 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27308032->27312127] (req 27308032->27312127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.603138:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb452a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493838720:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33057us (35878us total) trans 4294973986 rc 0/0 00000100:00100000:1.0:1789760781.603155:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6697 00000100:00100000:1.0:1789760781.603167:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791856820378 time 791911096551 state 0x2 ref 0 00000100:00100000:1.0:1789760781.607127:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760781.607365:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493838848 00000100:00100000:3.0:1789760781.607379:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493838848 00000100:00100000:3.0:1789760781.607450:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791856820378 time 791911096551 state 0x0 ref 1 00000100:00100000:3.0:1789760781.607470:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791911356967 time 791911096551 state 0x1 ref 1 00000100:00100000:3.0:1789760781.607531:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791911356967 time 791948565191 state 0x0 ref 1 00000100:00100000:3.0:1789760781.607545:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493838848:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760781.607613:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27312128, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xd809 00010000:00010000:3.0:1789760781.607623:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06c7e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.607644:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=791s v=(0 0) 00010000:00010000:3.0:1789760781.607649:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.607721:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06c7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.607743:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06c7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.642918:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973987 is committed 00002000:00010000:2.0:1789760781.643390:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27312128 -> 27316224 00002000:00010000:2.0:1789760781.643396:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760780 -> 1789760781 00002000:00010000:2.0:1789760781.643399:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760780 -> 1789760781 00010000:00010000:2.0:1789760781.643416:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06c7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.643431:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06c7e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.643441:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06c7e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.643449:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760781.643457:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06c7e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.643469:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06c7e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.643496:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06c7e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.643504:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06c7e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27312128->27316223] (req 27312128->27316223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.643630:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493838848:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36080us (36512us total) trans 4294973987 rc 0/0 00000100:00100000:2.0:1789760781.643643:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6698 00000100:00100000:2.0:1789760781.643655:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791911356967 time 791948565191 state 0x2 ref 0 00000100:00100000:0.0:1789760781.647543:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760781.647636:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493838976 00000100:00100000:3.0:1789760781.647649:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493838976 00000100:00100000:3.0:1789760781.647734:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791911356967 time 791948565191 state 0x0 ref 1 00000100:00100000:3.0:1789760781.647753:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791948825607 time 791948565191 state 0x1 ref 1 00000100:00100000:3.0:1789760781.647830:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791948825607 time 791988865135 state 0x0 ref 1 00000100:00100000:3.0:1789760781.647849:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493838976:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760781.647918:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27316224, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd809 00010000:00010000:3.0:1789760781.647928:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c85 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.647946:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=791s v=(0 0) 00010000:00010000:3.0:1789760781.647951:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.647988:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.648007:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.687643:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973988 is committed 00002000:00010000:3.0:1789760781.689469:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27316224 -> 27320320 00010000:00010000:3.0:1789760781.689490:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c85 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.689504:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c85 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.689514:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c85 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.689522:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760781.689528:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c85 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.689536:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c85 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.689558:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c85 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.689565:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c85 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27316224->27320319] (req 27316224->27320319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.689666:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493838976:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41817us (42131us total) trans 4294973988 rc 0/0 00000100:00100000:3.0:1789760781.689677:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6699 00000100:00100000:3.0:1789760781.689687:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791948825607 time 791988865135 state 0x2 ref 0 00000100:00100000:2.0:1789760781.694295:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760781.694683:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493839104 00000100:00100000:1.0:1789760781.694697:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493839104 00000100:00100000:1.0:1789760781.694781:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791948825607 time 791988865135 state 0x0 ref 1 00000100:00100000:1.0:1789760781.694801:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791989125551 time 791988865135 state 0x1 ref 1 00000100:00100000:1.0:1789760781.694839:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791989125551 time 792035873673 state 0x0 ref 1 00000100:00100000:1.0:1789760781.694856:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb453800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493839104:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760781.694918:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27320320, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd809 00010000:00010000:1.0:1789760781.694929:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06c8c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.694950:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=792s v=(0 0) 00010000:00010000:1.0:1789760781.694956:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.694997:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06c8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.695015:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06c8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760781.732443:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27320320 -> 27324416 00010000:00010000:1.0:1789760781.732463:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06c8c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.732477:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06c8c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.732487:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06c8c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.732496:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760781.732501:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06c8c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.732511:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06c8c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.732531:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06c8c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.732539:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06c8c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27320320->27324415] (req 27320320->27324415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.732633:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb453800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493839104:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37777us (38349us total) trans 4294973989 rc 0/0 00000100:00100000:1.0:1789760781.732642:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6700 00000100:00100000:1.0:1789760781.732653:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791989125551 time 792035873673 state 0x2 ref 0 00000001:00080000:0.0:1789760781.732913:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973989 is committed 00000100:00100000:2.0:1789760781.736658:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760781.736841:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493839232 00000100:00100000:0.0:1789760781.736853:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493839232 00000100:00100000:0.0:1789760781.736908:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 791989125551 time 792035873673 state 0x0 ref 1 00000100:00100000:0.0:1789760781.736925:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792036134089 time 792035873673 state 0x1 ref 1 00000100:00100000:0.0:1789760781.736956:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792036134089 time 792077991122 state 0x0 ref 1 00000100:00100000:2.0:1789760781.742903:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493839232:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760781.742981:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27324416, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd809 00010000:00010000:2.0:1789760781.742991:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06c93 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.743006:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=792s v=(0 0) 00010000:00010000:2.0:1789760781.743011:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.743018:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06c93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.743031:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06c93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.807619:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973990 is committed 00002000:00010000:3.0:1789760781.810616:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27324416 -> 27328512 00010000:00010000:3.0:1789760781.810643:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06c93 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.810655:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06c93 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.810663:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06c93 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.810676:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760781.810681:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06c93 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.810690:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06c93 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.810720:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06c93 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.810728:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06c93 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27324416->27328511] (req 27324416->27328511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.811100:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493839232:12345-192.168.203.60@tcp:4:dd.0 Request processed in 74043us (74449us total) trans 4294973990 rc 0/0 00000100:00100000:3.0:1789760781.811111:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6701 00000100:00100000:3.0:1789760781.811120:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792036134089 time 792077991122 state 0x2 ref 0 00000100:00100000:2.0:1789760781.815077:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760781.815192:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493839360 00000100:00100000:3.0:1789760781.815201:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493839360 00000100:00100000:3.0:1789760781.815265:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792036134089 time 792077991122 state 0x0 ref 1 00000100:00100000:3.0:1789760781.815280:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792078251538 time 792077991122 state 0x1 ref 1 00000100:00100000:3.0:1789760781.815310:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792078251538 time 792156346146 state 0x0 ref 1 00000100:00100000:3.0:1789760781.815327:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d8b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493839360:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760781.815381:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27328512, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd809 00010000:00010000:3.0:1789760781.815392:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c9a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.815407:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=792s v=(0 0) 00010000:00010000:3.0:1789760781.815411:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.815417:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.815430:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.841626:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973991 is committed 00002000:00010000:2.0:1789760781.841746:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27328512 -> 27332608 00010000:00010000:2.0:1789760781.841762:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.841775:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c9a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.841784:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c9a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.841792:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760781.841797:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c9a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.841807:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c9a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.841826:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c9a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.841903:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06c9a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27328512->27332607] (req 27328512->27332607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.842164:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d8b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493839360:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26835us (27092us total) trans 4294973991 rc 0/0 00000100:00100000:0.0:1789760781.842184:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6702 00000100:00100000:0.0:1789760781.842195:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792078251538 time 792156346146 state 0x2 ref 0 00000100:00100000:2.0:1789760781.846813:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760781.846993:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493839488 00000100:00100000:3.0:1789760781.847005:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493839488 00000100:00100000:3.0:1789760781.847059:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792078251538 time 792156346146 state 0x0 ref 1 00000100:00100000:3.0:1789760781.847075:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792156606562 time 792156346146 state 0x1 ref 1 00000100:00100000:3.0:1789760781.847106:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792156606562 time 792188141526 state 0x0 ref 1 00000100:00100000:3.0:1789760781.847118:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493839488:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760781.847170:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27332608, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd809 00010000:00010000:3.0:1789760781.847190:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06ca1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.847205:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=792s v=(0 0) 00010000:00010000:3.0:1789760781.847209:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.847215:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06ca1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.847233:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06ca1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.880045:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973992 is committed 00002000:00010000:3.0:1789760781.880294:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27332608 -> 27336704 00010000:00010000:3.0:1789760781.880310:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06ca1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.880323:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06ca1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.880332:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06ca1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.880339:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760781.880344:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06ca1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.880353:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06ca1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.880373:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06ca1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.880381:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06ca1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27332608->27336703] (req 27332608->27336703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.880474:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493839488:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33353us (33669us total) trans 4294973992 rc 0/0 00000100:00100000:3.0:1789760781.880484:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6703 00000100:00100000:3.0:1789760781.880493:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792156606562 time 792188141526 state 0x2 ref 0 00010000:00010000:1.0:1789760781.883170:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789760781.886912:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760781.887061:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493839616 00000100:00100000:1.0:1789760781.887071:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493839616 00000100:00100000:1.0:1789760781.887127:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792156606562 time 792188141526 state 0x0 ref 1 00000100:00100000:1.0:1789760781.887154:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792188401942 time 792188141526 state 0x1 ref 1 00000100:00100000:1.0:1789760781.887191:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792188401942 time 792228226605 state 0x0 ref 1 00000100:00100000:1.0:1789760781.887204:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493839616:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760781.887259:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27336704, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd809 00010000:00010000:1.0:1789760781.887268:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06ca8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.887283:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=792s v=(0 0) 00010000:00010000:1.0:1789760781.887287:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.887294:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06ca8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.887307:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06ca8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.935874:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973993 is committed 00002000:00010000:0.0:1789760781.936196:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27336704 -> 27340800 00010000:00010000:0.0:1789760781.936211:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06ca8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.936223:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06ca8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.936296:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06ca8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.936304:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760781.936309:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06ca8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.936319:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06ca8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.936339:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06ca8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.936347:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06ca8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27336704->27340799] (req 27336704->27340799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.936476:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493839616:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49269us (49640us total) trans 4294973993 rc 0/0 00000100:00100000:0.0:1789760781.936487:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6704 00000100:00100000:0.0:1789760781.936497:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792188401942 time 792228226605 state 0x2 ref 0 00000100:00100000:0.0:1789760781.940309:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760781.940400:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493839744 00000100:00100000:1.0:1789760781.940410:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493839744 00000100:00100000:1.0:1789760781.940468:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792188401942 time 792228226605 state 0x0 ref 1 00000100:00100000:1.0:1789760781.940484:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792228487021 time 792228226605 state 0x1 ref 1 00000100:00100000:1.0:1789760781.940518:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792228487021 time 792281553169 state 0x0 ref 1 00000100:00100000:1.0:1789760781.940532:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493839744:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760781.940589:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27340800, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd809 00010000:00010000:1.0:1789760781.940599:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06caf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.940614:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=792s v=(0 0) 00010000:00010000:1.0:1789760781.940618:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.940624:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06caf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.940639:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06caf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760781.976549:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27340800 -> 27344896 00010000:00010000:3.0:1789760781.976567:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06caf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.976582:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06caf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.976592:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06caf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.976600:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760781.976605:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06caf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.976615:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06caf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.976641:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06caf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.976649:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06caf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27340800->27344895] (req 27340800->27344895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.976741:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493839744:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36208us (36438us total) trans 4294973994 rc 0/0 00000100:00100000:3.0:1789760781.976751:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6705 00000100:00100000:3.0:1789760781.976761:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792228487021 time 792281553169 state 0x2 ref 0 00000001:00080000:2.0:1789760781.979554:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973994 is committed 00000100:00100000:2.0:1789760781.985192:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760781.985439:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493839872 00000100:00100000:0.0:1789760781.985450:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493839872 00000100:00100000:0.0:1789760781.985505:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792228487021 time 792281553169 state 0x0 ref 1 00000100:00100000:0.0:1789760781.985521:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792281813585 time 792281553169 state 0x1 ref 1 00000100:00100000:0.0:1789760781.985636:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792281813585 time 792326670474 state 0x0 ref 1 00000100:00100000:0.0:1789760781.985650:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493839872:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760781.985699:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27344896, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd809 00010000:00010000:0.0:1789760781.985709:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e06cb6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.985725:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=792s v=(0 0) 00010000:00010000:0.0:1789760781.985729:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.985736:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e06cb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.985750:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e06cb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760782.034066:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27344896 -> 27348992 00010000:00010000:3.0:1789760782.034086:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e06cb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.034098:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e06cb6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.034106:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e06cb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.034113:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760782.034117:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e06cb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.034131:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e06cb6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.034160:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e06cb6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.034167:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e06cb6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27344896->27348991] (req 27344896->27348991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.034283:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493839872:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48630us (49105us total) trans 4294973995 rc 0/0 00000100:00100000:3.0:1789760782.034293:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6706 00000100:00100000:3.0:1789760782.034302:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792281813585 time 792326670474 state 0x2 ref 0 00000001:00080000:2.0:1789760782.034867:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973995 is committed 00000100:00100000:0.0:1789760782.041845:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760782.041919:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493840000 00000100:00100000:3.0:1789760782.041928:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493840000 00000100:00100000:3.0:1789760782.042009:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792281813585 time 792326670474 state 0x0 ref 1 00000100:00100000:3.0:1789760782.042023:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792326930890 time 792326670474 state 0x1 ref 1 00000100:00100000:3.0:1789760782.042052:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792326930890 time 792383087609 state 0x0 ref 1 00000100:00100000:3.0:1789760782.042063:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493840000:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760782.042106:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27348992, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd809 00010000:00010000:3.0:1789760782.042115:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06cbd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.042131:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=792s v=(0 0) 00010000:00010000:3.0:1789760782.042135:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.042141:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06cbd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.042155:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06cbd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.071484:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973996 is committed 00002000:00010000:0.0:1789760782.071914:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27348992 -> 27353088 00010000:00010000:0.0:1789760782.071929:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06cbd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.071947:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06cbd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.071956:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06cbd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.071997:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760782.072001:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06cbd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.072010:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06cbd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.072030:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06cbd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.072038:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06cbd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27348992->27353087] (req 27348992->27353087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.072129:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493840000:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30062us (30287us total) trans 4294973996 rc 0/0 00000100:00100000:0.0:1789760782.072140:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6707 00000100:00100000:0.0:1789760782.072151:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792326930890 time 792383087609 state 0x2 ref 0 00000100:00100000:0.0:1789760782.075068:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760782.076305:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493840128 00000100:00100000:2.0:1789760782.076317:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493840128 00000100:00100000:2.0:1789760782.076447:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792326930890 time 792383087609 state 0x0 ref 1 00000100:00100000:2.0:1789760782.076463:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792383348025 time 792383087609 state 0x1 ref 1 00000100:00100000:2.0:1789760782.076493:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792383348025 time 792417527929 state 0x0 ref 1 00000100:00100000:2.0:1789760782.076508:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493840128:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760782.076559:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27353088, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd809 00010000:00010000:2.0:1789760782.076567:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06cc4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.076582:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=792s v=(0 0) 00010000:00010000:2.0:1789760782.076587:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.076594:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06cc4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.076608:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06cc4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.108412:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973997 is committed 00002000:00010000:0.0:1789760782.108622:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27353088 -> 27357184 00010000:00010000:0.0:1789760782.108637:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06cc4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.108649:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06cc4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.108658:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06cc4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.108665:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760782.108669:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06cc4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.108678:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06cc4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.108702:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06cc4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.108710:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06cc4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27353088->27357183] (req 27353088->27357183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.108804:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493840128:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32295us (33739us total) trans 4294973997 rc 0/0 00000100:00100000:0.0:1789760782.108814:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6708 00000100:00100000:0.0:1789760782.108829:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792383348025 time 792417527929 state 0x2 ref 0 00000100:00100000:2.0:1789760782.112474:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760782.112614:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493840256 00000100:00100000:1.0:1789760782.112625:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493840256 00000100:00100000:1.0:1789760782.112674:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792383348025 time 792417527929 state 0x0 ref 1 00000100:00100000:1.0:1789760782.112684:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792417788345 time 792417527929 state 0x1 ref 1 00000100:00100000:1.0:1789760782.112709:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792417788345 time 792453744171 state 0x0 ref 1 00000100:00100000:1.0:1789760782.112722:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fadb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493840256:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760782.112765:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27357184, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd809 00010000:00010000:1.0:1789760782.112772:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06ccb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.112785:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=792s v=(0 0) 00010000:00010000:1.0:1789760782.112788:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.112792:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06ccb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.112802:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06ccb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.136261:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973998 is committed 00002000:00010000:1.0:1789760782.136495:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27357184 -> 27361280 00010000:00010000:1.0:1789760782.136511:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06ccb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.136523:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06ccb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.136532:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06ccb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.136539:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760782.136544:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06ccb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.136552:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06ccb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.136577:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06ccb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.136585:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06ccb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27357184->27361279] (req 27357184->27361279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.136735:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fadb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493840256:12345-192.168.203.60@tcp:4:dd.0 Request processed in 24010us (24263us total) trans 4294973998 rc 0/0 00000100:00100000:1.0:1789760782.136745:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6709 00000100:00100000:1.0:1789760782.136755:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792417788345 time 792453744171 state 0x2 ref 0 00000100:00100000:2.0:1789760782.141795:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760782.141903:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493840384 00000100:00100000:3.0:1789760782.141912:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493840384 00000100:00100000:3.0:1789760782.142123:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792417788345 time 792453744171 state 0x0 ref 1 00000100:00100000:3.0:1789760782.142141:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792454004587 time 792453744171 state 0x1 ref 1 00000100:00100000:2.0:1789760782.142465:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792454004587 time 792483498271 state 0x0 ref 1 00000100:00100000:2.0:1789760782.142482:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c89fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493840384:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760782.142532:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27361280, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd809 00010000:00010000:2.0:1789760782.142541:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06cd2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.142556:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=792s v=(0 0) 00010000:00010000:2.0:1789760782.142560:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.142566:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06cd2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.142581:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06cd2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.166093:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973999 is committed 00002000:00010000:2.0:1789760782.166243:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27361280 -> 27365376 00010000:00010000:2.0:1789760782.166259:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06cd2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.166273:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06cd2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.166286:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06cd2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.166293:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760782.166298:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06cd2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.166307:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06cd2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.166326:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06cd2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.166334:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06cd2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27361280->27365375] (req 27361280->27365375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.166423:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c89fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493840384:12345-192.168.203.60@tcp:4:dd.0 Request processed in 23940us (24642us total) trans 4294973999 rc 0/0 00000100:00100000:2.0:1789760782.166432:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6710 00000100:00100000:2.0:1789760782.166441:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792454004587 time 792483498271 state 0x2 ref 0 00000100:00100000:0.0:1789760782.174511:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760782.174603:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493840512 00000100:00100000:3.0:1789760782.174614:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493840512 00000100:00100000:3.0:1789760782.174668:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792454004587 time 792483498271 state 0x0 ref 1 00000100:00100000:3.0:1789760782.174684:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792483758687 time 792483498271 state 0x1 ref 1 00000100:00100000:3.0:1789760782.174716:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792483758687 time 792515751735 state 0x0 ref 1 00000100:00100000:3.0:1789760782.174728:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493840512:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760782.174781:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27365376, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd809 00010000:00010000:3.0:1789760782.174790:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06cd9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.174813:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=792s v=(0 0) 00010000:00010000:3.0:1789760782.174817:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.174824:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06cd9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.174838:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06cd9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.223767:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974000 is committed 00002000:00010000:1.0:1789760782.223877:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27365376 -> 27369472 00010000:00010000:1.0:1789760782.223891:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06cd9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.223904:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06cd9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.223914:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06cd9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.223921:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760782.223926:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06cd9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.223940:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06cd9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.223995:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06cd9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.224004:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06cd9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27365376->27369471] (req 27365376->27369471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.224087:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493840512:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49356us (49583us total) trans 4294974000 rc 0/0 00000100:00100000:1.0:1789760782.224098:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6711 00000100:00100000:1.0:1789760782.224108:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792483758687 time 792515751735 state 0x2 ref 0 00000100:00100000:0.0:1789760782.230598:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760782.230697:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493840640 00000100:00100000:2.0:1789760782.230706:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493840640 00000100:00100000:2.0:1789760782.230761:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792483758687 time 792515751735 state 0x0 ref 1 00000100:00100000:2.0:1789760782.230777:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792516012151 time 792515751735 state 0x1 ref 1 00000100:00100000:2.0:1789760782.230843:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792516012151 time 792571878740 state 0x0 ref 1 00000100:00100000:2.0:1789760782.230855:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493840640:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760782.230901:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27369472, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd809 00010000:00010000:2.0:1789760782.230910:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06ce0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.230925:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=792s v=(0 0) 00010000:00010000:2.0:1789760782.230930:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.230937:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06ce0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.230951:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06ce0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.275197:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974001 is committed 00002000:00010000:3.0:1789760782.275434:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27369472 -> 27373568 00010000:00010000:3.0:1789760782.275451:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06ce0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.275466:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06ce0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.275475:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06ce0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.275483:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760782.275488:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06ce0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.275498:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06ce0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.275517:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06ce0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.275526:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06ce0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27369472->27373567] (req 27369472->27373567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.275620:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493840640:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44762us (45027us total) trans 4294974001 rc 0/0 00000100:00100000:3.0:1789760782.275630:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6712 00000100:00100000:3.0:1789760782.275639:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792516012151 time 792571878740 state 0x2 ref 0 00000100:00100000:0.0:1789760782.278518:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760782.278641:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493840768 00000100:00100000:1.0:1789760782.278720:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493840768 00000100:00100000:1.0:1789760782.278800:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792516012151 time 792571878740 state 0x0 ref 1 00000100:00100000:1.0:1789760782.278818:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792572139156 time 792571878740 state 0x1 ref 1 00000100:00100000:1.0:1789760782.278898:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792572139156 time 792619932529 state 0x0 ref 1 00000100:00100000:1.0:1789760782.278912:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493840768:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760782.279026:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27373568, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd809 00010000:00010000:1.0:1789760782.279037:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06ce7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.279057:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=792s v=(0 0) 00010000:00010000:1.0:1789760782.279062:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.279070:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06ce7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.279087:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06ce7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.318270:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974002 is committed 00002000:00010000:2.0:1789760782.318664:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27373568 -> 27377664 00010000:00010000:2.0:1789760782.318683:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06ce7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.318699:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06ce7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.318709:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06ce7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.318717:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760782.318723:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06ce7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.318733:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06ce7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.318758:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06ce7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.318767:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06ce7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27373568->27377663] (req 27373568->27377663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.318923:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493840768:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40006us (40412us total) trans 4294974002 rc 0/0 00000100:00100000:2.0:1789760782.318935:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6713 00000100:00100000:2.0:1789760782.318947:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792572139156 time 792619932529 state 0x2 ref 0 00000100:00100000:1.0:1789760782.321523:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760782.321717:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493840896 00000100:00100000:0.0:1789760782.321730:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493840896 00000100:00100000:0.0:1789760782.321803:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792572139156 time 792619932529 state 0x0 ref 1 00000100:00100000:0.0:1789760782.321823:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792620192945 time 792619932529 state 0x1 ref 1 00000100:00100000:0.0:1789760782.321856:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792620192945 time 792662891457 state 0x0 ref 1 00000100:00100000:0.0:1789760782.321873:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493840896:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760782.321940:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27377664, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd809 00010000:00010000:0.0:1789760782.321951:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06cee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.322128:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=792s v=(0 0) 00010000:00010000:0.0:1789760782.322134:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.322142:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06cee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.322160:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06cee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.372838:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974003 is committed 00002000:00010000:3.0:1789760782.373192:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27377664 -> 27381760 00010000:00010000:3.0:1789760782.373220:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06cee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.373236:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06cee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.373247:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06cee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.373255:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760782.373261:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06cee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.373272:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06cee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.373302:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06cee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.373310:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06cee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27377664->27381759] (req 27377664->27381759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.373442:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493840896:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51564us (51925us total) trans 4294974003 rc 0/0 00000100:00100000:3.0:1789760782.373455:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6714 00000100:00100000:3.0:1789760782.373467:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792620192945 time 792662891457 state 0x2 ref 0 00000100:00100000:0.0:1789760782.376693:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760782.376786:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493841024 00000100:00100000:3.0:1789760782.376801:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493841024 00000100:00100000:3.0:1789760782.376889:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792620192945 time 792662891457 state 0x0 ref 1 00000100:00100000:3.0:1789760782.376907:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792663151873 time 792662891457 state 0x1 ref 1 00000100:00100000:3.0:1789760782.376942:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792663151873 time 792717977156 state 0x0 ref 1 00000100:00100000:3.0:1789760782.376957:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493841024:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760782.377067:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27381760, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd809 00010000:00010000:3.0:1789760782.377078:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06cf5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.377097:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=792s v=(0 0) 00010000:00010000:3.0:1789760782.377103:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.377110:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06cf5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.377124:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06cf5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.416064:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974004 is committed 00002000:00010000:3.0:1789760782.416212:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27381760 -> 27385856 00010000:00010000:3.0:1789760782.416239:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06cf5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.416255:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06cf5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.416265:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06cf5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.416272:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760782.416278:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06cf5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.416289:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06cf5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.416314:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06cf5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.416320:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06cf5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27381760->27385855] (req 27381760->27385855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.416446:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493841024:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39484us (39761us total) trans 4294974004 rc 0/0 00000100:00100000:3.0:1789760782.416458:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6715 00000100:00100000:3.0:1789760782.416470:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792663151873 time 792717977156 state 0x2 ref 0 00000100:00100000:0.0:1789760782.420024:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760782.420105:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493841152 00000100:00100000:3.0:1789760782.420119:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493841152 00000100:00100000:3.0:1789760782.420201:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792663151873 time 792717977156 state 0x0 ref 1 00000100:00100000:3.0:1789760782.420220:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792718237572 time 792717977156 state 0x1 ref 1 00000100:00100000:3.0:1789760782.420425:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792718237572 time 792761458608 state 0x0 ref 1 00000100:00100000:3.0:1789760782.420440:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493841152:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760782.420506:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27385856, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd809 00010000:00010000:3.0:1789760782.420517:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06cfc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.420537:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=792s v=(0 0) 00010000:00010000:3.0:1789760782.420542:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.420550:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06cfc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.420566:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06cfc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.462574:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974005 is committed 00002000:00010000:0.0:1789760782.462858:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27385856 -> 27389952 00010000:00010000:0.0:1789760782.462879:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06cfc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.462893:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06cfc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.462903:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06cfc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.462910:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760782.462915:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06cfc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.462926:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06cfc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.462949:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06cfc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.462956:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06cfc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27385856->27389951] (req 27385856->27389951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.463094:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493841152:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42650us (43125us total) trans 4294974005 rc 0/0 00000100:00100000:0.0:1789760782.463107:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6716 00000100:00100000:0.0:1789760782.463119:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792718237572 time 792761458608 state 0x2 ref 0 00000100:00100000:0.0:1789760782.465773:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760782.465923:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493841280 00000100:00100000:2.0:1789760782.465936:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493841280 00000100:00100000:2.0:1789760782.466053:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792718237572 time 792761458608 state 0x0 ref 1 00000100:00100000:2.0:1789760782.466072:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792761719024 time 792761458608 state 0x1 ref 1 00000100:00100000:2.0:1789760782.466109:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792761719024 time 792807144231 state 0x0 ref 1 00000100:00100000:2.0:1789760782.466123:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493841280:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760782.466210:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27389952, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd809 00010000:00010000:2.0:1789760782.466222:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06d03 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.466247:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=792s v=(0 0) 00010000:00010000:2.0:1789760782.466253:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.466262:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06d03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.466278:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06d03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.516782:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974006 is committed 00002000:00010000:2.0:1789760782.516997:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27389952 -> 27394048 00010000:00010000:2.0:1789760782.517019:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06d03 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.517037:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06d03 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.517048:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06d03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.517056:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760782.517062:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06d03 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.517072:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06d03 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.517099:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06d03 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.517108:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06d03 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27389952->27394047] (req 27389952->27394047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.517240:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493841280:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51111us (51472us total) trans 4294974006 rc 0/0 00000100:00100000:2.0:1789760782.517253:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6717 00000100:00100000:2.0:1789760782.517265:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792761719024 time 792807144231 state 0x2 ref 0 00000100:00100000:2.0:1789760782.523784:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760782.523865:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493841408 00000100:00100000:0.0:1789760782.523880:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493841408 00000100:00100000:0.0:1789760782.523955:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792761719024 time 792807144231 state 0x0 ref 1 00000100:00100000:0.0:1789760782.524010:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792807404647 time 792807144231 state 0x1 ref 1 00000100:00100000:0.0:1789760782.524051:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792807404647 time 792865085201 state 0x0 ref 1 00000100:00100000:0.0:1789760782.524066:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493841408:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760782.524132:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27394048, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd809 00010000:00010000:0.0:1789760782.524143:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d0a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.524162:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=792s v=(0 0) 00010000:00010000:0.0:1789760782.524168:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.524189:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.524214:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760782.573807:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27394048 -> 27398144 00010000:00010000:0.0:1789760782.573835:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d0a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.573849:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d0a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.573858:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.573866:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760782.573872:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d0a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.573881:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d0a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.573905:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d0a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.573913:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d0a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27394048->27398143] (req 27394048->27398143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.574080:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493841408:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50010us (50305us total) trans 4294974007 rc 0/0 00000100:00100000:0.0:1789760782.574093:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6718 00000100:00100000:0.0:1789760782.574104:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792807404647 time 792865085201 state 0x2 ref 0 00000001:00080000:2.0:1789760782.574506:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974007 is committed 00000100:00100000:0.0:1789760782.581113:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760782.581292:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493841536 00000100:00100000:3.0:1789760782.581307:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493841536 00000100:00100000:3.0:1789760782.581383:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792807404647 time 792865085201 state 0x0 ref 1 00000100:00100000:3.0:1789760782.581403:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792865345617 time 792865085201 state 0x1 ref 1 00000100:00100000:3.0:1789760782.581448:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792865345617 time 792922482830 state 0x0 ref 1 00000100:00100000:3.0:1789760782.581464:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493841536:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760782.581554:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27398144, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xd809 00010000:00010000:3.0:1789760782.581564:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06d11 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.581583:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=792s v=(0 0) 00010000:00010000:3.0:1789760782.581588:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.581596:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06d11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.581613:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06d11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.614744:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974008 is committed 00002000:00010000:0.0:1789760782.614875:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27398144 -> 27402240 00002000:00010000:0.0:1789760782.614881:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760781 -> 1789760782 00002000:00010000:0.0:1789760782.614884:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760781 -> 1789760782 00010000:00010000:0.0:1789760782.614903:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06d11 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.614917:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06d11 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.614927:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06d11 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.614936:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760782.614942:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06d11 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.614953:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06d11 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.615013:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06d11 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.615022:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06d11 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27398144->27402239] (req 27398144->27402239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.615237:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493841536:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33758us (34133us total) trans 4294974008 rc 0/0 00000100:00100000:0.0:1789760782.615252:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6719 00000100:00100000:0.0:1789760782.615264:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792865345617 time 792922482830 state 0x2 ref 0 00000100:00100000:0.0:1789760782.623894:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760782.624364:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493841664 00000100:00100000:3.0:1789760782.624379:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493841664 00000100:00100000:3.0:1789760782.624458:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792865345617 time 792922482830 state 0x0 ref 1 00000100:00100000:3.0:1789760782.624478:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792922743246 time 792922482830 state 0x1 ref 1 00000100:00100000:0.0:1789760782.624822:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792922743246 time 792965852770 state 0x0 ref 1 00000100:00100000:0.0:1789760782.624844:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493841664:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760782.624914:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27402240, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd809 00010000:00010000:0.0:1789760782.624925:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d18 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.624945:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=792s v=(0 0) 00010000:00010000:0.0:1789760782.624951:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.625028:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.625059:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760782.686553:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27402240 -> 27406336 00010000:00010000:0.0:1789760782.686582:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d18 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.686598:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d18 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.686607:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.686615:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760782.686621:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d18 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.686632:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d18 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.686669:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d18 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.686678:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d18 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27402240->27406335] (req 27402240->27406335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.686805:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493841664:12345-192.168.203.60@tcp:4:dd.0 Request processed in 61958us (62917us total) trans 4294974009 rc 0/0 00000100:00100000:0.0:1789760782.686821:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6720 00000100:00100000:0.0:1789760782.686834:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792922743246 time 792965852770 state 0x2 ref 0 00000001:00080000:1.0:1789760782.686951:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974009 is committed 00000100:00100000:2.0:1789760782.690473:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760782.690750:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493841792 00000100:00100000:1.0:1789760782.690762:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493841792 00000100:00100000:1.0:1789760782.690842:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792922743246 time 792965852770 state 0x0 ref 1 00000100:00100000:1.0:1789760782.690861:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792966113186 time 792965852770 state 0x1 ref 1 00000100:00100000:0.0:1789760782.691015:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792966113186 time 793032044674 state 0x0 ref 1 00000100:00100000:0.0:1789760782.691034:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493841792:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760782.691113:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27406336, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd809 00010000:00010000:0.0:1789760782.691124:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06d1f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.691145:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=793s v=(0 0) 00010000:00010000:0.0:1789760782.691150:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.691158:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06d1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.691184:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06d1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.744396:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974010 is committed 00002000:00010000:0.0:1789760782.744546:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27406336 -> 27410432 00010000:00010000:0.0:1789760782.744565:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06d1f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.744580:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06d1f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.744595:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06d1f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.744673:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760782.744679:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06d1f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.744690:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06d1f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.744724:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06d1f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.744733:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06d1f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27406336->27410431] (req 27406336->27410431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.744862:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493841792:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53825us (54399us total) trans 4294974010 rc 0/0 00000100:00100000:0.0:1789760782.744873:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6721 00000100:00100000:0.0:1789760782.744885:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792966113186 time 793032044674 state 0x2 ref 0 00000100:00100000:2.0:1789760782.749209:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760782.754468:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493841920 00000100:00100000:3.0:1789760782.754492:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493841920 00000100:00100000:2.0:1789760782.754689:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 792966113186 time 793032044674 state 0x0 ref 1 00000100:00100000:2.0:1789760782.754711:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793032305090 time 793032044674 state 0x1 ref 1 00000100:00100000:2.0:1789760782.754748:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793032305090 time 793095783146 state 0x0 ref 1 00000100:00100000:2.0:1789760782.754766:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8555880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493841920:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760782.754832:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27410432, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd809 00010000:00010000:2.0:1789760782.754843:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06d26 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.754865:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=793s v=(0 0) 00010000:00010000:2.0:1789760782.754871:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.754878:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06d26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.754894:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06d26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760782.808697:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27410432 -> 27414528 00010000:00010000:2.0:1789760782.808725:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06d26 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.808740:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06d26 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.808750:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06d26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.808758:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760782.808764:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06d26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.808775:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06d26 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.808799:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06d26 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.808809:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06d26 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27410432->27414527] (req 27410432->27414527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.809221:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8555880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493841920:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54451us (60020us total) trans 4294974011 rc 0/0 00000100:00100000:2.0:1789760782.809242:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6722 00000100:00100000:2.0:1789760782.809254:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793032305090 time 793095783146 state 0x2 ref 0 00000001:00080000:2.0:1789760782.809304:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974011 is committed 00000100:00100000:2.0:1789760782.813257:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760782.813354:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493842048 00000100:00100000:1.0:1789760782.813366:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493842048 00000100:00100000:1.0:1789760782.813443:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793032305090 time 793095783146 state 0x0 ref 1 00000100:00100000:1.0:1789760782.813598:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793096043562 time 793095783146 state 0x1 ref 1 00000100:00100000:1.0:1789760782.813627:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793096043562 time 793154662876 state 0x0 ref 1 00000100:00100000:1.0:1789760782.813643:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493842048:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760782.813703:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27414528, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd809 00010000:00010000:1.0:1789760782.813714:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06d2d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.813733:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=793s v=(0 0) 00010000:00010000:1.0:1789760782.813738:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.813745:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06d2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.813762:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06d2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.873334:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974012 is committed 00002000:00010000:2.0:1789760782.873426:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27414528 -> 27418624 00010000:00010000:2.0:1789760782.873445:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06d2d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.873460:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06d2d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.873470:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06d2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.873478:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760782.873483:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06d2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.873499:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06d2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.873522:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06d2d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.873530:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06d2d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27414528->27418623] (req 27414528->27418623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.873656:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493842048:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60010us (60406us total) trans 4294974012 rc 0/0 00000100:00100000:2.0:1789760782.873668:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6723 00000100:00100000:2.0:1789760782.873680:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793096043562 time 793154662876 state 0x2 ref 0 00000100:00100000:3.0:1789760782.876536:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760782.876749:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493842176 00000100:00100000:0.0:1789760782.876763:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493842176 00000100:00100000:0.0:1789760782.876836:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793096043562 time 793154662876 state 0x0 ref 1 00000100:00100000:0.0:1789760782.876854:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793154923292 time 793154662876 state 0x1 ref 1 00000100:00100000:0.0:1789760782.876885:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793154923292 time 793217920664 state 0x0 ref 1 00000100:00100000:0.0:1789760782.876900:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493842176:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760782.876993:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27418624, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd809 00010000:00010000:0.0:1789760782.877004:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d34 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.877023:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=793s v=(0 0) 00010000:00010000:0.0:1789760782.877028:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.877036:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.877054:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.908547:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760782.956330:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974013 is committed 00002000:00010000:3.0:1789760782.956512:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27418624 -> 27422720 00010000:00010000:3.0:1789760782.956530:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.956543:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d34 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.956552:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.956559:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760782.956565:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.956574:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d34 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.956594:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d34 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.956602:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e06d34 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27418624->27422719] (req 27418624->27422719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.956701:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493842176:12345-192.168.203.60@tcp:4:dd.0 Request processed in 79799us (80173us total) trans 4294974013 rc 0/0 00000100:00100000:3.0:1789760782.956713:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6724 00000100:00100000:3.0:1789760782.956723:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793154923292 time 793217920664 state 0x2 ref 0 00000100:00100000:0.0:1789760782.961485:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760782.961589:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493842304 00000100:00100000:3.0:1789760782.961598:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493842304 00000100:00100000:3.0:1789760782.961708:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793154923292 time 793217920664 state 0x0 ref 1 00000100:00100000:3.0:1789760782.961728:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793218181080 time 793217920664 state 0x1 ref 1 00000100:00100000:3.0:1789760782.961758:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793218181080 time 793302793935 state 0x0 ref 1 00000100:00100000:3.0:1789760782.961771:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493842304:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760782.961829:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27422720, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd809 00010000:00010000:3.0:1789760782.961842:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06d3b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.961921:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=793s v=(0 0) 00010000:00010000:3.0:1789760782.961926:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.961933:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06d3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.961949:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06d3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.999909:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974014 is committed 00002000:00010000:0.0:1789760783.001938:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27422720 -> 27426816 00010000:00010000:0.0:1789760783.002706:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06d3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.002728:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06d3b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.002737:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06d3b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.002819:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760783.002825:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06d3b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.002836:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06d3b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.002863:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06d3b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.002873:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06d3b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27422720->27426815] (req 27422720->27426815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.003040:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493842304:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41262us (41557us total) trans 4294974014 rc 0/0 00000100:00100000:0.0:1789760783.003053:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6725 00000100:00100000:0.0:1789760783.003066:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793218181080 time 793302793935 state 0x2 ref 0 00000100:00100000:0.0:1789760783.009803:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760783.010402:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493842432 00000100:00100000:2.0:1789760783.010416:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493842432 00000100:00100000:2.0:1789760783.010567:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793218181080 time 793302793935 state 0x0 ref 1 00000100:00100000:2.0:1789760783.010586:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793303054351 time 793302793935 state 0x1 ref 1 00000100:00100000:2.0:1789760783.010622:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793303054351 time 793351656418 state 0x0 ref 1 00000100:00100000:2.0:1789760783.010639:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493842432:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760783.010704:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27426816, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd809 00010000:00010000:2.0:1789760783.010714:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06d42 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.010733:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=793s v=(0 0) 00010000:00010000:2.0:1789760783.010749:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.010758:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06d42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.010774:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06d42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760783.012374:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789760783.053833:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27426816 -> 27430912 00010000:00010000:3.0:1789760783.053861:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06d42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.053877:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06d42 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.053886:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06d42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.053895:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760783.053900:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06d42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.053911:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06d42 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.053936:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06d42 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.053945:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06d42 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27426816->27430911] (req 27426816->27430911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.054106:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493842432:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43454us (44302us total) trans 4294974015 rc 0/0 00000100:00100000:3.0:1789760783.054118:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6726 00000100:00100000:3.0:1789760783.054130:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793303054351 time 793351656418 state 0x2 ref 0 00000001:00080000:0.0:1789760783.055416:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974015 is committed 00000100:00100000:2.0:1789760783.063348:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760783.063531:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493842560 00000100:00100000:1.0:1789760783.063544:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493842560 00000100:00100000:1.0:1789760783.063622:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793303054351 time 793351656418 state 0x0 ref 1 00000100:00100000:1.0:1789760783.063712:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793351916834 time 793351656418 state 0x1 ref 1 00000100:00100000:1.0:1789760783.063795:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793351916834 time 793404829420 state 0x0 ref 1 00000100:00100000:1.0:1789760783.063811:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4529c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493842560:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760783.063876:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27430912, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd809 00010000:00010000:1.0:1789760783.063889:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06d49 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.063914:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=793s v=(0 0) 00010000:00010000:1.0:1789760783.063920:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.063928:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06d49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.063945:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06d49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760783.092048:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27430912 -> 27435008 00000001:00080000:0.0:1789760783.092077:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974016 is committed 00010000:00010000:3.0:1789760783.092077:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06d49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.092092:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06d49 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.092102:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06d49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.092110:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760783.092116:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06d49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.092127:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06d49 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.092153:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06d49 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.092161:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06d49 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27430912->27435007] (req 27430912->27435007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.092568:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4529c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493842560:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28751us (29228us total) trans 4294974016 rc 0/0 00000100:00100000:3.0:1789760783.092584:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6727 00000100:00100000:3.0:1789760783.092597:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793351916834 time 793404829420 state 0x2 ref 0 00000100:00100000:1.0:1789760783.095957:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760783.096471:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493842688 00000100:00100000:0.0:1789760783.096484:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493842688 00000100:00100000:0.0:1789760783.096560:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793351916834 time 793404829420 state 0x0 ref 1 00000100:00100000:0.0:1789760783.096580:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793405089836 time 793404829420 state 0x1 ref 1 00000100:00100000:0.0:1789760783.096663:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793405089836 time 793437697013 state 0x0 ref 1 00000100:00100000:0.0:1789760783.096680:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493842688:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760783.096748:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27435008, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd809 00010000:00010000:0.0:1789760783.096759:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06d50 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.096778:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=793s v=(0 0) 00010000:00010000:0.0:1789760783.096783:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.096790:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06d50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.096807:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06d50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.145575:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974017 is committed 00002000:00010000:0.0:1789760783.145712:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27435008 -> 27439104 00010000:00010000:0.0:1789760783.145732:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06d50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.145746:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06d50 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.145756:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06d50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.145764:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760783.145769:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06d50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.145780:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06d50 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.145805:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06d50 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.145812:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06d50 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27435008->27439103] (req 27435008->27439103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.146019:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493842688:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49334us (50069us total) trans 4294974017 rc 0/0 00000100:00100000:0.0:1789760783.146031:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6728 00000100:00100000:0.0:1789760783.146042:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793405089836 time 793437697013 state 0x2 ref 0 00000100:00100000:0.0:1789760783.148866:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760783.149204:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493842816 00000100:00100000:2.0:1789760783.149217:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493842816 00000100:00100000:2.0:1789760783.149294:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793405089836 time 793437697013 state 0x0 ref 1 00000100:00100000:2.0:1789760783.149313:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793437957429 time 793437697013 state 0x1 ref 1 00000100:00100000:2.0:1789760783.149349:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793437957429 time 793490384008 state 0x0 ref 1 00000100:00100000:2.0:1789760783.149362:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493842816:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760783.149425:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27439104, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd809 00010000:00010000:2.0:1789760783.149435:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06d57 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.149455:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=793s v=(0 0) 00010000:00010000:2.0:1789760783.149460:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.149477:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06d57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.149494:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06d57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.208057:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974018 is committed 00002000:00010000:0.0:1789760783.208200:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27439104 -> 27443200 00010000:00010000:0.0:1789760783.208334:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06d57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.208350:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06d57 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.208360:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06d57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.208374:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760783.208381:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06d57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.208391:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06d57 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.208417:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06d57 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.208425:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06d57 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27439104->27443199] (req 27439104->27443199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.208558:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493842816:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59190us (59699us total) trans 4294974018 rc 0/0 00000100:00100000:0.0:1789760783.208571:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6729 00000100:00100000:0.0:1789760783.208583:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793437957429 time 793490384008 state 0x2 ref 0 00000100:00100000:2.0:1789760783.212689:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760783.212947:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493842944 00000100:00100000:3.0:1789760783.213152:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493842944 00000100:00100000:3.0:1789760783.213243:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793437957429 time 793490384008 state 0x0 ref 1 00000100:00100000:3.0:1789760783.213261:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793490644424 time 793490384008 state 0x1 ref 1 00000100:00100000:3.0:1789760783.213302:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793490644424 time 793554337045 state 0x0 ref 1 00000100:00100000:3.0:1789760783.213316:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493842944:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760783.213380:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27443200, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd809 00010000:00010000:3.0:1789760783.213390:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d5e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.213409:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=793s v=(0 0) 00010000:00010000:3.0:1789760783.213414:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.213422:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.213442:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.246554:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974019 is committed 00002000:00010000:3.0:1789760783.246607:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27443200 -> 27447296 00010000:00010000:3.0:1789760783.246633:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.246691:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d5e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.246703:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d5e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.246716:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760783.246847:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d5e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.246859:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d5e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.246886:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d5e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.246895:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d5e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27443200->27447295] (req 27443200->27447295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.247080:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493842944:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33757us (34396us total) trans 4294974019 rc 0/0 00000100:00100000:3.0:1789760783.247094:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6730 00000100:00100000:3.0:1789760783.247107:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793490644424 time 793554337045 state 0x2 ref 0 00000100:00100000:2.0:1789760783.253260:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760783.253329:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493843072 00000100:00100000:3.0:1789760783.253343:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493843072 00000100:00100000:3.0:1789760783.253425:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793490644424 time 793554337045 state 0x0 ref 1 00000100:00100000:3.0:1789760783.253446:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793554597461 time 793554337045 state 0x1 ref 1 00000100:00100000:3.0:1789760783.253481:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793554597461 time 793594515981 state 0x0 ref 1 00000100:00100000:3.0:1789760783.253496:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493843072:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760783.253561:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27447296, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd809 00010000:00010000:3.0:1789760783.253572:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d65 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.253593:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=793s v=(0 0) 00010000:00010000:3.0:1789760783.253599:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.253607:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.253624:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.304611:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974020 is committed 00002000:00010000:3.0:1789760783.306476:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27447296 -> 27451392 00010000:00010000:3.0:1789760783.306515:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.306533:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d65 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.306544:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d65 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.306553:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760783.306559:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d65 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.306570:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d65 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.306596:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d65 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.306604:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d65 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27447296->27451391] (req 27447296->27451391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.307102:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493843072:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53599us (53847us total) trans 4294974020 rc 0/0 00000100:00100000:3.0:1789760783.307121:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6731 00000100:00100000:3.0:1789760783.307133:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793554597461 time 793594515981 state 0x2 ref 0 00000100:00100000:1.0:1789760783.313501:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760783.313670:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493843200 00000100:00100000:0.0:1789760783.313684:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493843200 00000100:00100000:0.0:1789760783.313758:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793554597461 time 793594515981 state 0x0 ref 1 00000100:00100000:0.0:1789760783.313778:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793594776397 time 793594515981 state 0x1 ref 1 00000100:00100000:0.0:1789760783.313816:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793594776397 time 793654850715 state 0x0 ref 1 00000100:00100000:0.0:1789760783.313830:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493843200:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760783.313897:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27451392, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd809 00010000:00010000:0.0:1789760783.313907:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06d6c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.313927:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=793s v=(0 0) 00010000:00010000:0.0:1789760783.313933:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.313940:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06d6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.313957:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06d6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.386315:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974021 is committed 00002000:00010000:1.0:1789760783.386511:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27451392 -> 27455488 00010000:00010000:1.0:1789760783.386532:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06d6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.386546:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06d6c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.386555:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06d6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.386563:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760783.386569:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06d6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.386580:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06d6c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.386606:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06d6c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.386614:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06d6c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27451392->27455487] (req 27451392->27455487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.386748:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493843200:12345-192.168.203.60@tcp:4:dd.0 Request processed in 72912us (73254us total) trans 4294974021 rc 0/0 00000100:00100000:1.0:1789760783.386760:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6732 00000100:00100000:1.0:1789760783.386771:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793594776397 time 793654850715 state 0x2 ref 0 00000100:00100000:2.0:1789760783.402131:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760783.402254:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493843328 00000100:00100000:0.0:1789760783.402267:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493843328 00000100:00100000:0.0:1789760783.402343:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793594776397 time 793654850715 state 0x0 ref 1 00000100:00100000:0.0:1789760783.402363:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793655111131 time 793654850715 state 0x1 ref 1 00000100:00100000:0.0:1789760783.402401:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793655111131 time 793743436367 state 0x0 ref 1 00000100:00100000:0.0:1789760783.402414:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493843328:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760783.402483:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27455488, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd809 00010000:00010000:0.0:1789760783.402494:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06d73 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.402520:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=793s v=(0 0) 00010000:00010000:0.0:1789760783.402525:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.402533:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06d73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.402550:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06d73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.464341:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760783.464494:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493843456 00000100:00100000:3.0:1789760783.464509:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493843456 00000100:00100000:3.0:1789760783.464584:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 415 00000100:00100000:3.0:1789760783.464596:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d48450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2367:x1876699493843456:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:3.0:1789760783.464700:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d48450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2367:x1876699493843456:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 106us (376us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760783.464708:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 415 00000100:00100000:2.0:1789760783.470031:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760783.470089:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760783.470129:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493843584 00000100:00100000:2.0:1789760783.470143:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493843584 00000100:00100000:2.0:1789760783.470296:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493843712 00000100:00100000:2.0:1789760783.470301:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493843712 00000100:00100000:2.0:1789760783.470337:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 522 00000100:00100000:2.0:1789760783.470348:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2367:x1876699493843584:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:2.0:1789760783.470445:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2367:x1876699493843584:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 98us (431us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760783.470452:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 522 00000100:00100000:2.0:1789760783.470472:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 523 00000100:00100000:2.0:1789760783.470477:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2368:x1876699493843712:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:2.0:1789760783.470503:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc47aed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2368:x1876699493843712:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 25us (414us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760783.470509:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 523 00000100:00080000:2.0:1789760783.470527:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:1.0:1789760783.484111:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974022 is committed 00002000:00010000:0.0:1789760783.484358:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27455488 -> 27459584 00010000:00010000:0.0:1789760783.484378:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06d73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.484393:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06d73 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.484403:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06d73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.484410:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760783.484416:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06d73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.484426:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06d73 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.484452:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06d73 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.484461:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06d73 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27455488->27459583] (req 27455488->27459583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.484640:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493843328:12345-192.168.203.60@tcp:4:dd.0 Request processed in 82219us (82518us total) trans 4294974022 rc 0/0 00000100:00100000:0.0:1789760783.484652:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6733 00000100:00100000:0.0:1789760783.484665:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793655111131 time 793743436367 state 0x2 ref 0 00000100:00100000:3.0:1789760783.492490:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760783.492758:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493843840 00000100:00100000:0.0:1789760783.492772:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493843840 00000100:00100000:0.0:1789760783.492853:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793655111131 time 793743436367 state 0x0 ref 1 00000100:00100000:0.0:1789760783.492873:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793743696783 time 793743436367 state 0x1 ref 1 00000100:00100000:0.0:1789760783.492909:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793743696783 time 793833944638 state 0x0 ref 1 00000100:00100000:0.0:1789760783.492924:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493843840:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760783.493020:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27459584, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd809 00010000:00010000:0.0:1789760783.493032:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06d7a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.493053:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=793s v=(0 0) 00010000:00010000:0.0:1789760783.493058:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.493065:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06d7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.493087:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06d7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760783.562712:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27459584 -> 27463680 00010000:00010000:1.0:1789760783.562741:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06d7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.562759:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06d7a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.562770:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06d7a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.562778:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760783.562783:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06d7a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.562794:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06d7a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.562821:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06d7a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.562829:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06d7a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27459584->27463679] (req 27459584->27463679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.563009:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493843840:12345-192.168.203.60@tcp:4:dd.0 Request processed in 70032us (70480us total) trans 4294974023 rc 0/0 00000100:00100000:1.0:1789760783.563023:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6734 00000100:00100000:1.0:1789760783.563036:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793743696783 time 793833944638 state 0x2 ref 0 00000001:00080000:1.0:1789760783.563088:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974023 is committed 00000100:00100000:0.0:1789760783.566538:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760783.566802:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493843968 00000100:00100000:3.0:1789760783.566815:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493843968 00000100:00100000:3.0:1789760783.566885:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793743696783 time 793833944638 state 0x0 ref 1 00000100:00100000:3.0:1789760783.566902:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793834205054 time 793833944638 state 0x1 ref 1 00000100:00100000:3.0:1789760783.567130:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793834205054 time 793908163807 state 0x0 ref 1 00000100:00100000:3.0:1789760783.567145:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493843968:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760783.567460:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27463680, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd809 00010000:00010000:2.0:1789760783.567475:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d81 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.567495:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=793s v=(0 0) 00010000:00010000:2.0:1789760783.567500:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.567508:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.567524:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.603363:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974024 is committed 00002000:00010000:2.0:1789760783.603482:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27463680 -> 27467776 00010000:00010000:2.0:1789760783.603503:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.603520:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d81 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.603530:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.603538:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760783.603545:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.603555:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d81 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.603585:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d81 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.603594:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06d81 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27463680->27467775] (req 27463680->27467775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.603776:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493843968:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36624us (37242us total) trans 4294974024 rc 0/0 00000100:00100000:2.0:1789760783.603788:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6735 00000100:00100000:2.0:1789760783.603801:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793834205054 time 793908163807 state 0x2 ref 0 00000100:00100000:3.0:1789760783.608454:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760783.608562:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493844096 00000100:00100000:1.0:1789760783.608575:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493844096 00000100:00100000:1.0:1789760783.608669:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793834205054 time 793908163807 state 0x0 ref 1 00000100:00100000:1.0:1789760783.608692:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793908424223 time 793908163807 state 0x1 ref 1 00000100:00100000:1.0:1789760783.608731:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793908424223 time 793949765979 state 0x0 ref 1 00000100:00100000:1.0:1789760783.608746:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493844096:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760783.608820:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27467776, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xd809 00010000:00010000:1.0:1789760783.608832:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06d88 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.608852:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=793s v=(0 0) 00010000:00010000:1.0:1789760783.608857:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.608866:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06d88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.608885:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06d88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.658400:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974025 is committed 00002000:00010000:0.0:1789760783.658513:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27467776 -> 27471872 00002000:00010000:0.0:1789760783.658518:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760782 -> 1789760783 00002000:00010000:0.0:1789760783.658521:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760782 -> 1789760783 00010000:00010000:0.0:1789760783.658539:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06d88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.658554:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06d88 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.658564:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06d88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.658573:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760783.658579:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06d88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.658590:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06d88 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.658615:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06d88 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.658624:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06d88 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27467776->27471871] (req 27467776->27471871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.658868:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493844096:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50116us (50424us total) trans 4294974025 rc 0/0 00000100:00100000:0.0:1789760783.658882:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6736 00000100:00100000:0.0:1789760783.658895:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793908424223 time 793949765979 state 0x2 ref 0 00000100:00100000:2.0:1789760783.665309:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760783.665458:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493844224 00000100:00100000:0.0:1789760783.665470:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493844224 00000100:00100000:0.0:1789760783.665543:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793908424223 time 793949765979 state 0x0 ref 1 00000100:00100000:0.0:1789760783.665561:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793950026395 time 793949765979 state 0x1 ref 1 00000100:00100000:0.0:1789760783.665597:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793950026395 time 794006632393 state 0x0 ref 1 00000100:00100000:0.0:1789760783.665610:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493844224:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760783.665690:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27471872, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd809 00010000:00010000:0.0:1789760783.665701:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06d8f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.665720:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=794s v=(0 0) 00010000:00010000:0.0:1789760783.665725:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.665731:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06d8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.665755:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06d8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760783.704604:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27471872 -> 27475968 00010000:00010000:0.0:1789760783.704629:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06d8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.704713:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06d8f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.704723:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06d8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.704731:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760783.704737:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06d8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.704747:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06d8f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.704771:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06d8f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.704779:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e06d8f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27471872->27475967] (req 27471872->27475967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.704910:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493844224:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39295us (39610us total) trans 4294974026 rc 0/0 00000100:00100000:0.0:1789760783.704923:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6737 00000100:00100000:0.0:1789760783.704934:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793950026395 time 794006632393 state 0x2 ref 0 00000001:00080000:1.0:1789760783.705072:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974026 is committed 00000100:00100000:2.0:1789760783.708865:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760783.711395:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493844352 00000100:00100000:0.0:1789760783.711415:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493844352 00000100:00100000:0.0:1789760783.711502:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 793950026395 time 794006632393 state 0x0 ref 1 00000100:00100000:0.0:1789760783.711524:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794006892809 time 794006632393 state 0x1 ref 1 00000100:00100000:0.0:1789760783.711567:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794006892809 time 794052603060 state 0x0 ref 1 00000100:00100000:0.0:1789760783.711583:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493844352:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760783.711718:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27475968, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd809 00010000:00010000:0.0:1789760783.711730:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06d96 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.711750:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=794s v=(0 0) 00010000:00010000:0.0:1789760783.711755:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.711762:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06d96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.711779:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06d96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.748074:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974027 is committed 00002000:00010000:0.0:1789760783.748199:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27475968 -> 27480064 00010000:00010000:0.0:1789760783.748219:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06d96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.748235:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06d96 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.748246:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06d96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.748254:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760783.748259:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06d96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.748269:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06d96 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.748294:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06d96 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.748302:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06d96 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27475968->27480063] (req 27475968->27480063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.748710:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493844352:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37124us (39853us total) trans 4294974027 rc 0/0 00000100:00100000:0.0:1789760783.748727:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6738 00000100:00100000:0.0:1789760783.748740:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794006892809 time 794052603060 state 0x2 ref 0 00000100:00100000:2.0:1789760783.751853:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760783.752014:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493844480 00000100:00100000:1.0:1789760783.752029:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493844480 00000100:00100000:1.0:1789760783.752107:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794006892809 time 794052603060 state 0x0 ref 1 00000100:00100000:1.0:1789760783.752126:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794052863476 time 794052603060 state 0x1 ref 1 00000100:00100000:1.0:1789760783.752171:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794052863476 time 794093196650 state 0x0 ref 1 00000100:00100000:1.0:1789760783.752186:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493844480:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760783.752285:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27480064, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd809 00010000:00010000:1.0:1789760783.752297:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06d9d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.752318:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=794s v=(0 0) 00010000:00010000:1.0:1789760783.752323:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.752331:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06d9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.752461:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06d9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.789676:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974028 is committed 00002000:00010000:0.0:1789760783.789790:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27480064 -> 27484160 00010000:00010000:0.0:1789760783.789805:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06d9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.789817:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06d9d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.789826:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06d9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.789832:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760783.789837:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06d9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.789846:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06d9d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.789865:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06d9d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.789872:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06d9d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27480064->27484159] (req 27480064->27484159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.790012:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493844480:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37823us (38168us total) trans 4294974028 rc 0/0 00000100:00100000:0.0:1789760783.790023:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6739 00000100:00100000:0.0:1789760783.790032:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794052863476 time 794093196650 state 0x2 ref 0 00000100:00100000:2.0:1789760783.797620:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760783.797734:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493844608 00000100:00100000:1.0:1789760783.797754:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493844608 00000100:00100000:1.0:1789760783.797819:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794052863476 time 794093196650 state 0x0 ref 1 00000100:00100000:1.0:1789760783.797835:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794093457066 time 794093196650 state 0x1 ref 1 00000100:00100000:1.0:1789760783.797869:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794093457066 time 794138904216 state 0x0 ref 1 00000100:00100000:1.0:1789760783.797883:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493844608:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760783.797935:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27484160, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd809 00010000:00010000:1.0:1789760783.797944:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06da4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.799617:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=794s v=(0 0) 00010000:00010000:1.0:1789760783.799622:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.799630:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06da4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.799647:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06da4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760783.848138:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27484160 -> 27488256 00010000:00010000:0.0:1789760783.848157:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06da4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.848178:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06da4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.848187:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06da4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.848195:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760783.848200:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06da4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.848210:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06da4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.848237:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06da4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.848250:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06da4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27484160->27488255] (req 27484160->27488255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.848344:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493844608:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50460us (50728us total) trans 4294974029 rc 0/0 00000100:00100000:0.0:1789760783.848354:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6740 00000100:00100000:0.0:1789760783.848364:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794093457066 time 794138904216 state 0x2 ref 0 00000001:00080000:2.0:1789760783.849375:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974029 is committed 00000100:00100000:2.0:1789760783.854615:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760783.854755:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493844736 00000100:00100000:0.0:1789760783.854764:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493844736 00000100:00100000:0.0:1789760783.855225:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794093457066 time 794138904216 state 0x0 ref 1 00000100:00100000:0.0:1789760783.855246:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794139164632 time 794138904216 state 0x1 ref 1 00000100:00100000:1.0:1789760783.855429:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794139164632 time 794196461527 state 0x0 ref 1 00000100:00100000:1.0:1789760783.855447:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493844736:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760783.855498:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27488256, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd809 00010000:00010000:1.0:1789760783.855507:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06dab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.855524:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=794s v=(0 0) 00010000:00010000:1.0:1789760783.855528:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.855535:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06dab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.855550:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06dab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760783.892653:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27488256 -> 27492352 00010000:00010000:0.0:1789760783.892675:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06dab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.892688:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06dab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.892697:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06dab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.892704:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760783.892709:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06dab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.892719:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06dab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.892739:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06dab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.892747:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06dab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27488256->27492351] (req 27488256->27492351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.892839:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493844736:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37393us (38228us total) trans 4294974030 rc 0/0 00000100:00100000:0.0:1789760783.892849:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6741 00000100:00100000:0.0:1789760783.892859:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794139164632 time 794196461527 state 0x2 ref 0 00000001:00080000:3.0:1789760783.893290:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974030 is committed 00000100:00100000:3.0:1789760783.895943:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760783.896048:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493844864 00000100:00100000:0.0:1789760783.896062:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493844864 00000100:00100000:0.0:1789760783.896121:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794139164632 time 794196461527 state 0x0 ref 1 00000100:00100000:0.0:1789760783.896138:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794196721943 time 794196461527 state 0x1 ref 1 00000100:00100000:0.0:1789760783.896299:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794196721943 time 794237332971 state 0x0 ref 1 00000100:00100000:0.0:1789760783.896315:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493844864:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760783.896367:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27492352, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd809 00010000:00010000:0.0:1789760783.896376:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06db2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.896770:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=794s v=(0 0) 00010000:00010000:0.0:1789760783.896775:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.896782:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06db2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.896796:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06db2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.930123:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760783.942002:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974031 is committed 00002000:00010000:2.0:1789760783.942157:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27492352 -> 27496448 00010000:00010000:2.0:1789760783.942189:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06db2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.942206:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06db2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.942294:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06db2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.942303:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760783.942309:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06db2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.942319:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06db2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.942343:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06db2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.942352:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e06db2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27492352->27496447] (req 27492352->27496447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.942474:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493844864:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46155us (46538us total) trans 4294974031 rc 0/0 00000100:00100000:2.0:1789760783.942487:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6742 00000100:00100000:2.0:1789760783.942501:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794196721943 time 794237332971 state 0x2 ref 0 00000100:00100000:2.0:1789760783.948140:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760783.948260:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493844992 00000100:00100000:1.0:1789760783.948274:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493844992 00000100:00100000:1.0:1789760783.948349:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794196721943 time 794237332971 state 0x0 ref 1 00000100:00100000:1.0:1789760783.948369:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794237593387 time 794237332971 state 0x1 ref 1 00000100:00100000:1.0:1789760783.948454:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794237593387 time 794289488969 state 0x0 ref 1 00000100:00100000:1.0:1789760783.948470:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493844992:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760783.948537:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27496448, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd809 00010000:00010000:1.0:1789760783.948549:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06db9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.948568:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=794s v=(0 0) 00010000:00010000:1.0:1789760783.948573:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.948580:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06db9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.948596:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06db9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760784.008248:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27496448 -> 27500544 00010000:00010000:1.0:1789760784.008280:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06db9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.008297:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06db9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.008308:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06db9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.008317:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760784.008323:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06db9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.008334:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06db9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.008362:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06db9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.008370:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06db9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27496448->27500543] (req 27496448->27500543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.008509:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493844992:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60035us (60375us total) trans 4294974032 rc 0/0 00000100:00100000:1.0:1789760784.008522:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6743 00000100:00100000:1.0:1789760784.008535:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794237593387 time 794289488969 state 0x2 ref 0 00000001:00080000:3.0:1789760784.009389:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974032 is committed 00000100:00100000:0.0:1789760784.014552:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760784.015410:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493845120 00000100:00100000:1.0:1789760784.015424:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493845120 00000100:00100000:1.0:1789760784.015505:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794237593387 time 794289488969 state 0x0 ref 1 00000100:00100000:1.0:1789760784.015524:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794289749385 time 794289488969 state 0x1 ref 1 00000100:00100000:1.0:1789760784.015563:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794289749385 time 794356598111 state 0x0 ref 1 00000100:00100000:1.0:1789760784.015579:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493845120:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760784.015644:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27500544, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd809 00010000:00010000:1.0:1789760784.015655:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06dc0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.015675:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=794s v=(0 0) 00010000:00010000:1.0:1789760784.015682:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.015689:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06dc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.015706:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06dc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.060175:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974033 is committed 00002000:00010000:0.0:1789760784.060379:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27500544 -> 27504640 00010000:00010000:0.0:1789760784.060395:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06dc0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.060408:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06dc0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.060418:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06dc0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.060425:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760784.060430:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06dc0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.060440:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06dc0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.060464:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06dc0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.060472:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06dc0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27500544->27504639] (req 27500544->27504639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.060561:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493845120:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44980us (46017us total) trans 4294974033 rc 0/0 00000100:00100000:0.0:1789760784.060572:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6744 00000100:00100000:0.0:1789760784.060582:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794289749385 time 794356598111 state 0x2 ref 0 00000100:00100000:0.0:1789760784.064523:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760784.064643:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493845248 00000100:00100000:1.0:1789760784.064653:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493845248 00000100:00100000:1.0:1789760784.064705:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794289749385 time 794356598111 state 0x0 ref 1 00000100:00100000:1.0:1789760784.064721:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794356858527 time 794356598111 state 0x1 ref 1 00000100:00100000:1.0:1789760784.064797:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794356858527 time 794405831512 state 0x0 ref 1 00000100:00100000:1.0:1789760784.064809:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493845248:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760784.064859:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27504640, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd809 00010000:00010000:1.0:1789760784.064868:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06dc7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.064883:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=794s v=(0 0) 00010000:00010000:1.0:1789760784.064888:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.064894:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06dc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.064907:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06dc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.111997:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974034 is committed 00002000:00010000:2.0:1789760784.112098:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27504640 -> 27508736 00010000:00010000:2.0:1789760784.112113:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06dc7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.112131:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06dc7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.112141:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06dc7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.112148:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760784.112153:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06dc7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.112172:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06dc7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.112200:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06dc7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.112208:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e06dc7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27504640->27508735] (req 27504640->27508735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.112313:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493845248:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47501us (47794us total) trans 4294974034 rc 0/0 00000100:00100000:2.0:1789760784.112325:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6745 00000100:00100000:2.0:1789760784.112335:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794356858527 time 794405831512 state 0x2 ref 0 00000100:00100000:2.0:1789760784.116524:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760784.116713:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493845376 00000100:00100000:0.0:1789760784.116723:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493845376 00000100:00100000:0.0:1789760784.116778:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794356858527 time 794405831512 state 0x0 ref 1 00000100:00100000:0.0:1789760784.116792:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794406091928 time 794405831512 state 0x1 ref 1 00000100:00100000:0.0:1789760784.116826:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794406091928 time 794457861916 state 0x0 ref 1 00000100:00100000:0.0:1789760784.116838:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493845376:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760784.116887:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27508736, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd809 00010000:00010000:0.0:1789760784.116896:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06dce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.116916:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=794s v=(0 0) 00010000:00010000:0.0:1789760784.116920:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.116927:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06dce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.116941:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06dce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760784.164179:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27508736 -> 27512832 00010000:00010000:0.0:1789760784.164199:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06dce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.164220:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06dce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.164230:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06dce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.164238:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760784.164242:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06dce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.164252:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06dce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.164272:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06dce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.164280:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e06dce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27508736->27512831] (req 27508736->27512831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.164377:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493845376:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47537us (47870us total) trans 4294974035 rc 0/0 00000100:00100000:0.0:1789760784.164389:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6746 00000100:00100000:0.0:1789760784.164400:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794406091928 time 794457861916 state 0x2 ref 0 00000001:00080000:3.0:1789760784.164587:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974035 is committed 00000100:00100000:0.0:1789760784.172554:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760784.172687:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493845504 00000100:00100000:2.0:1789760784.172698:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493845504 00000100:00100000:2.0:1789760784.172761:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794406091928 time 794457861916 state 0x0 ref 1 00000100:00100000:2.0:1789760784.172777:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794458122332 time 794457861916 state 0x1 ref 1 00000100:00100000:2.0:1789760784.172864:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794458122332 time 794513898657 state 0x0 ref 1 00000100:00100000:2.0:1789760784.172878:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493845504:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760784.172939:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27512832, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd809 00010000:00010000:2.0:1789760784.172948:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06dd5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.173005:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=794s v=(0 0) 00010000:00010000:2.0:1789760784.173010:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.173016:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06dd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.173031:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06dd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.209018:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974036 is committed 00002000:00010000:3.0:1789760784.209171:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27512832 -> 27516928 00010000:00010000:3.0:1789760784.209186:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06dd5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.209198:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06dd5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.209206:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06dd5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.209224:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760784.209229:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06dd5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.209237:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06dd5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.209255:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06dd5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.209263:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06dd5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27512832->27516927] (req 27512832->27516927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.209356:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493845504:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36476us (36808us total) trans 4294974036 rc 0/0 00000100:00100000:3.0:1789760784.209369:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6747 00000100:00100000:3.0:1789760784.209378:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794458122332 time 794513898657 state 0x2 ref 0 00000100:00100000:2.0:1789760784.214545:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760784.214835:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493845632 00000100:00100000:1.0:1789760784.214847:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493845632 00000100:00100000:1.0:1789760784.214910:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794458122332 time 794513898657 state 0x0 ref 1 00000100:00100000:1.0:1789760784.214927:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794514159073 time 794513898657 state 0x1 ref 1 00000100:00100000:1.0:1789760784.215559:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794514159073 time 794555994332 state 0x0 ref 1 00000100:00100000:1.0:1789760784.215582:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493845632:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760784.215629:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27516928, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd809 00010000:00010000:1.0:1789760784.215639:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06ddc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.215653:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=794s v=(0 0) 00010000:00010000:1.0:1789760784.215657:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.215663:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06ddc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.215677:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06ddc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.252080:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974037 is committed 00000100:00100000:3.0:1789760784.252358:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc2886680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3303:1876699519260544:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760784.252418:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00010000:1.0:1789760784.252512:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27516928 -> 27521024 00010000:00010000:1.0:1789760784.252527:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06ddc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.252546:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06ddc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.252555:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06ddc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.252562:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760784.252566:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06ddc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.252575:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06ddc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.252592:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06ddc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.252600:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06ddc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27516928->27521023] (req 27516928->27521023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.252666:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493845632:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37088us (38130us total) trans 4294974037 rc 0/0 00000100:00100000:1.0:1789760784.252675:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6748 00000100:00100000:1.0:1789760784.252684:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794514159073 time 794555994332 state 0x2 ref 0 00000100:00100000:0.0:1789760784.252830:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519260544 00000100:00100000:0.0:1789760784.252837:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519260544 00000100:00100000:0.0:1789760784.252983:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 216 00000100:00100000:0.0:1789760784.252991:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876699519260544:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760784.253010:0:6690:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526712832 pending 0 free 3830448128 avail 3798990848 00000100:00100000:0.0:1789760784.253085:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876699519260544:12345-0@lo:13:osp-pre-0-0.0 Request processed in 95us (667us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760784.253092:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 216 00000100:00080000:2.0:1789760784.253417:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760784.253554:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc2886680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3303:1876699519260544:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760784.261037:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760784.261133:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493845760 00000100:00100000:1.0:1789760784.261143:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493845760 00000100:00100000:1.0:1789760784.261211:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794514159073 time 794555994332 state 0x0 ref 1 00000100:00100000:1.0:1789760784.261226:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794556254748 time 794555994332 state 0x1 ref 1 00000100:00100000:1.0:1789760784.261253:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794556254748 time 794602289293 state 0x0 ref 1 00000100:00100000:1.0:1789760784.261265:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493845760:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760784.261315:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27521024, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd809 00010000:00010000:1.0:1789760784.261324:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06de3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.261341:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=794s v=(0 0) 00010000:00010000:1.0:1789760784.261346:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.261353:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06de3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.261370:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06de3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.332185:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974038 is committed 00002000:00010000:3.0:1789760784.332831:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27521024 -> 27525120 00010000:00010000:3.0:1789760784.332855:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06de3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.332871:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06de3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.332882:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06de3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.332891:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760784.332897:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06de3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.332907:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06de3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.332931:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06de3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.332940:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06de3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27521024->27525119] (req 27521024->27525119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.333099:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493845760:12345-192.168.203.60@tcp:4:dd.0 Request processed in 71828us (72065us total) trans 4294974038 rc 0/0 00000100:00100000:3.0:1789760784.333112:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6749 00000100:00100000:3.0:1789760784.333125:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794556254748 time 794602289293 state 0x2 ref 0 00000100:00100000:0.0:1789760784.337028:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760784.337118:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493845888 00000100:00100000:2.0:1789760784.337132:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493845888 00000100:00100000:2.0:1789760784.337453:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794556254748 time 794602289293 state 0x0 ref 1 00000100:00100000:2.0:1789760784.337480:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794602549709 time 794602289293 state 0x1 ref 1 00000100:00100000:2.0:1789760784.337521:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794602549709 time 794678556248 state 0x0 ref 1 00000100:00100000:2.0:1789760784.337538:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493845888:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760784.338043:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27525120, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd809 00010000:00010000:2.0:1789760784.338056:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06dea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.338155:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=794s v=(0 0) 00010000:00010000:2.0:1789760784.338169:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.338177:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06dea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.338195:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06dea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760784.368516:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27525120 -> 27529216 00010000:00010000:2.0:1789760784.368542:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06dea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.368559:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06dea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.368569:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06dea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.368583:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760784.368590:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06dea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.368599:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06dea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.368622:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06dea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.368629:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e06dea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27525120->27529215] (req 27525120->27529215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.368752:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493845888:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31211us (31732us total) trans 4294974039 rc 0/0 00000100:00100000:2.0:1789760784.368764:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6750 00000100:00100000:2.0:1789760784.368777:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794602549709 time 794678556248 state 0x2 ref 0 00000001:00080000:3.0:1789760784.369463:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974039 is committed 00000100:00100000:1.0:1789760784.373239:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760784.373337:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493846016 00000100:00100000:3.0:1789760784.373350:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493846016 00000100:00100000:3.0:1789760784.373432:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794602549709 time 794678556248 state 0x0 ref 1 00000100:00100000:3.0:1789760784.373453:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794678816664 time 794678556248 state 0x1 ref 1 00000100:00100000:3.0:1789760784.373491:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794678816664 time 794714526140 state 0x0 ref 1 00000100:00100000:3.0:1789760784.373505:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493846016:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760784.373575:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27529216, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd809 00010000:00010000:3.0:1789760784.373587:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06df1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.373608:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=794s v=(0 0) 00010000:00010000:3.0:1789760784.373612:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.373619:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06df1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.373637:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06df1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.407770:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974040 is committed 00002000:00010000:0.0:1789760784.408028:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27529216 -> 27533312 00010000:00010000:0.0:1789760784.408050:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06df1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.408067:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06df1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.408085:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06df1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.408093:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760784.408099:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06df1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.408110:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06df1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.408137:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06df1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.408146:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e06df1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27529216->27533311] (req 27529216->27533311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.408396:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493846016:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34885us (35175us total) trans 4294974040 rc 0/0 00000100:00100000:0.0:1789760784.408410:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6751 00000100:00100000:0.0:1789760784.408424:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794678816664 time 794714526140 state 0x2 ref 0 00000100:00100000:0.0:1789760784.413074:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760784.413207:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493846144 00000100:00100000:3.0:1789760784.413237:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493846144 00000100:00100000:3.0:1789760784.413322:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794678816664 time 794714526140 state 0x0 ref 1 00000100:00100000:3.0:1789760784.413342:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794714786556 time 794714526140 state 0x1 ref 1 00000100:00100000:3.0:1789760784.413384:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794714786556 time 794754418613 state 0x0 ref 1 00000100:00100000:3.0:1789760784.413399:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb16450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493846144:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760784.413474:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27533312, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd809 00010000:00010000:3.0:1789760784.413485:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06df8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.413506:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=794s v=(0 0) 00010000:00010000:3.0:1789760784.413512:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.413520:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06df8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.413536:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06df8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.445859:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974041 is committed 00002000:00010000:2.0:1789760784.445920:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27533312 -> 27537408 00010000:00010000:2.0:1789760784.445940:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06df8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.445956:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06df8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.446036:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06df8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.446043:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760784.446047:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06df8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.446055:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06df8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.446076:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06df8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.446082:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e06df8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27533312->27537407] (req 27533312->27537407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.447601:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb16450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493846144:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34191us (34525us total) trans 4294974041 rc 0/0 00000100:00100000:2.0:1789760784.447617:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6752 00000100:00100000:2.0:1789760784.447629:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794714786556 time 794754418613 state 0x2 ref 0 00000100:00100000:2.0:1789760784.453360:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760784.454923:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493846272 00000100:00100000:1.0:1789760784.454939:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493846272 00000100:00100000:1.0:1789760784.455053:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794714786556 time 794754418613 state 0x0 ref 1 00000100:00100000:1.0:1789760784.455069:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794754679029 time 794754418613 state 0x1 ref 1 00000100:00100000:1.0:1789760784.455097:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794754679029 time 794796133015 state 0x0 ref 1 00000100:00100000:1.0:1789760784.455111:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493846272:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760784.456285:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27537408, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd809 00010000:00010000:1.0:1789760784.456300:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06dff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.456318:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=794s v=(0 0) 00010000:00010000:1.0:1789760784.456323:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.456330:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06dff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.456344:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06dff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.494774:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974042 is committed 00002000:00010000:0.0:1789760784.494990:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27537408 -> 27541504 00010000:00010000:0.0:1789760784.495012:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06dff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.495033:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06dff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.495043:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06dff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.495052:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760784.495058:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06dff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.495069:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06dff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.495097:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06dff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.495104:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e06dff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27537408->27541503] (req 27537408->27541503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.495252:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493846272:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40135us (41898us total) trans 4294974042 rc 0/0 00000100:00100000:0.0:1789760784.495263:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6753 00000100:00100000:0.0:1789760784.495276:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794754679029 time 794796133015 state 0x2 ref 0 00000100:00100000:1.0:1789760784.498274:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760784.498490:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493846400 00000100:00100000:2.0:1789760784.498505:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493846400 00000100:00100000:2.0:1789760784.498590:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794754679029 time 794796133015 state 0x0 ref 1 00000100:00100000:2.0:1789760784.498610:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794796393431 time 794796133015 state 0x1 ref 1 00000100:00100000:2.0:1789760784.498656:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794796393431 time 794839691357 state 0x0 ref 1 00000100:00100000:2.0:1789760784.498672:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493846400:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760784.498741:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27541504, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd809 00010000:00010000:2.0:1789760784.498752:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e06 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.498772:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=794s v=(0 0) 00010000:00010000:2.0:1789760784.498778:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.498786:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.498802:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760784.535843:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27541504 -> 27545600 00010000:00010000:2.0:1789760784.535875:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e06 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.535892:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e06 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.535902:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e06 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.535911:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760784.535917:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e06 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.535929:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e06 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.535989:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e06 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.535999:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e06 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27541504->27545599] (req 27541504->27545599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.536113:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493846400:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37437us (37850us total) trans 4294974043 rc 0/0 00000100:00100000:2.0:1789760784.536126:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6754 00000100:00100000:2.0:1789760784.536139:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794796393431 time 794839691357 state 0x2 ref 0 00000001:00080000:0.0:1789760784.536308:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974043 is committed 00000100:00100000:2.0:1789760784.539113:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760784.539308:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493846528 00000100:00100000:0.0:1789760784.539321:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493846528 00000100:00100000:0.0:1789760784.539392:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794796393431 time 794839691357 state 0x0 ref 1 00000100:00100000:0.0:1789760784.539410:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794839951773 time 794839691357 state 0x1 ref 1 00000100:00100000:0.0:1789760784.539446:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794839951773 time 794880481080 state 0x0 ref 1 00000100:00100000:0.0:1789760784.539459:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493846528:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760784.539524:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27545600, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd809 00010000:00010000:0.0:1789760784.539535:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06e0d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.539555:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=794s v=(0 0) 00010000:00010000:0.0:1789760784.539560:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.539567:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06e0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.539584:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06e0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.575996:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974044 is committed 00002000:00010000:2.0:1789760784.576175:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27545600 -> 27549696 00010000:00010000:2.0:1789760784.576195:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06e0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.576219:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06e0d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.576235:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06e0d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.576243:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760784.576249:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06e0d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.576259:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06e0d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.576284:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06e0d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.576294:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06e0d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27545600->27549695] (req 27545600->27549695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.576458:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493846528:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36994us (37353us total) trans 4294974044 rc 0/0 00000100:00100000:2.0:1789760784.576470:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6755 00000100:00100000:2.0:1789760784.576482:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794839951773 time 794880481080 state 0x2 ref 0 00000100:00100000:3.0:1789760784.588400:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760784.588566:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493846656 00000100:00100000:2.0:1789760784.588579:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493846656 00000100:00100000:2.0:1789760784.588664:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794839951773 time 794880481080 state 0x0 ref 1 00000100:00100000:2.0:1789760784.588689:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794880741496 time 794880481080 state 0x1 ref 1 00000100:00100000:2.0:1789760784.588728:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794880741496 time 794929763207 state 0x0 ref 1 00000100:00100000:2.0:1789760784.588743:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493846656:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760784.588808:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27549696, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xd809 00010000:00010000:2.0:1789760784.588820:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e14 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.588840:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=794s v=(0 0) 00010000:00010000:2.0:1789760784.588845:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.588852:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.588869:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.631135:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974045 is committed 00002000:00010000:3.0:1789760784.631286:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27549696 -> 27553792 00002000:00010000:3.0:1789760784.631292:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760783 -> 1789760784 00002000:00010000:3.0:1789760784.631296:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760783 -> 1789760784 00010000:00010000:3.0:1789760784.631314:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.631332:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e14 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.631342:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e14 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.631350:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760784.631356:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e14 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.631367:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e14 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.631395:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e14 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.631410:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e14 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27549696->27553791] (req 27549696->27553791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.631593:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493846656:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42845us (43199us total) trans 4294974045 rc 0/0 00000100:00100000:3.0:1789760784.631608:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6756 00000100:00100000:3.0:1789760784.631620:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794880741496 time 794929763207 state 0x2 ref 0 00000100:00100000:0.0:1789760784.635417:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760784.635583:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493846784 00000100:00100000:3.0:1789760784.635597:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493846784 00000100:00100000:3.0:1789760784.635748:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794880741496 time 794929763207 state 0x0 ref 1 00000100:00100000:3.0:1789760784.635767:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794930023623 time 794929763207 state 0x1 ref 1 00000100:00100000:3.0:1789760784.635843:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794930023623 time 794976877088 state 0x0 ref 1 00000100:00100000:3.0:1789760784.635858:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493846784:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760784.635930:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27553792, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:3.0:1789760784.635944:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06e1b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.636013:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=794s v=(0 0) 00010000:00010000:3.0:1789760784.636020:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.636027:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06e1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.636053:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06e1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.677882:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974046 is committed 00002000:00010000:3.0:1789760784.678125:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27553792 -> 27557888 00010000:00010000:3.0:1789760784.678146:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06e1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.678193:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06e1b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.678205:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06e1b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.678223:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760784.678229:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06e1b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.678241:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06e1b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.678268:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06e1b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.678276:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06e1b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27553792->27557887] (req 27553792->27557887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.678402:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493846784:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42541us (42996us total) trans 4294974046 rc 0/0 00000100:00100000:3.0:1789760784.678414:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6757 00000100:00100000:3.0:1789760784.678427:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794930023623 time 794976877088 state 0x2 ref 0 00000100:00100000:2.0:1789760784.682110:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760784.682474:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493846912 00000100:00100000:0.0:1789760784.682489:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493846912 00000100:00100000:0.0:1789760784.682568:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794930023623 time 794976877088 state 0x0 ref 1 00000100:00100000:0.0:1789760784.682588:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794977137504 time 794976877088 state 0x1 ref 1 00000100:00100000:0.0:1789760784.682735:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794977137504 time 795023768962 state 0x0 ref 1 00000100:00100000:0.0:1789760784.682751:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493846912:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760784.682824:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27557888, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:0.0:1789760784.682835:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06e22 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.682854:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=795s v=(0 0) 00010000:00010000:0.0:1789760784.682859:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.682867:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06e22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.682896:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06e22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.725087:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974047 is committed 00002000:00010000:0.0:1789760784.725233:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27557888 -> 27561984 00010000:00010000:0.0:1789760784.725254:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06e22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.725269:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06e22 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.725280:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06e22 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.725288:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760784.725295:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06e22 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.725305:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06e22 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.725330:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06e22 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.725338:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06e22 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27557888->27561983] (req 27557888->27561983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.725527:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493846912:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42772us (43424us total) trans 4294974047 rc 0/0 00000100:00100000:0.0:1789760784.725541:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6758 00000100:00100000:0.0:1789760784.725553:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794977137504 time 795023768962 state 0x2 ref 0 00000100:00100000:2.0:1789760784.729179:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760784.729359:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493847040 00000100:00100000:0.0:1789760784.729372:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493847040 00000100:00100000:0.0:1789760784.729446:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 794977137504 time 795023768962 state 0x0 ref 1 00000100:00100000:0.0:1789760784.729464:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795024029378 time 795023768962 state 0x1 ref 1 00000100:00100000:0.0:1789760784.729500:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795024029378 time 795070534404 state 0x0 ref 1 00000100:00100000:0.0:1789760784.729514:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493847040:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760784.729582:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27561984, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:0.0:1789760784.729594:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06e29 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.729615:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=795s v=(0 0) 00010000:00010000:0.0:1789760784.729620:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.729628:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06e29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.729772:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06e29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.758903:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974048 is committed 00002000:00010000:0.0:1789760784.759065:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27561984 -> 27566080 00010000:00010000:0.0:1789760784.759086:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06e29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.759101:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06e29 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.759110:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06e29 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.759117:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760784.759123:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06e29 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.759133:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06e29 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.759174:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06e29 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.759182:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06e29 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27561984->27566079] (req 27561984->27566079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.759367:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493847040:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29847us (30208us total) trans 4294974048 rc 0/0 00000100:00100000:0.0:1789760784.759380:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6759 00000100:00100000:0.0:1789760784.759392:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795024029378 time 795070534404 state 0x2 ref 0 00000100:00100000:0.0:1789760784.763135:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760784.763452:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc2885f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3305:1876699519260800:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.763520:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760784.763580:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620c7aa5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519261312:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.763614:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760784.763657:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760784.763663:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc2887b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519260928:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.763689:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760784.763732:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519261312 00000100:00100000:1.0:1789760784.763740:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519261312 00000100:00100000:1.0:1789760784.763806:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519260928 00000100:00100000:1.0:1789760784.763810:0:6687:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519260928 00000100:00100000:1.0:1789760784.763839:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 186 00000100:00100000:1.0:1789760784.763850:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c3094700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519261312:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.763912:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c3094700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519261312:12345-0@lo:400:kworker.0 Request processed in 64us (297us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760784.763920:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 186 00000100:00100000:1.0:1789760784.763937:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 187 00000100:00100000:1.0:1789760784.763943:0:6687:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519260928:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.763998:0:6687:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519260928:12345-0@lo:400:kworker.0 Request processed in 55us (307us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760784.764004:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 187 00000100:00080000:1.0:1789760784.764021:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789760784.764058:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc2887b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519260928:0@lo:400:kworker.0 00000100:00100000:1.0:1789760784.764071:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620c7aa5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519261312:0@lo:400:kworker.0 00000100:00100000:0.0:1789760784.764178:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc2884a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3303:1876699519261056:0@lo:400:kworker.0 00000100:00100000:0.0:1789760784.764224:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760784.764247:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1789760784.764254:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc2885c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3303:1876699519260672:0@lo:400:kworker.0 00000100:00100000:0.0:1789760784.764285:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760784.764311:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc2886680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3303:1876699519261184:0@lo:400:kworker.0 00000100:00100000:0.0:1789760784.764334:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760784.764365:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519260800 00000100:00100000:0.0:1789760784.764370:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519260800 00000100:00100000:0.0:1789760784.764403:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519261056 00000100:00100000:0.0:1789760784.764406:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519261056 00000100:00100000:0.0:1789760784.764430:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519261184 00000100:00100000:0.0:1789760784.764432:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519261184 00000100:00100000:0.0:1789760784.764447:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 524 00000100:00100000:0.0:1789760784.764454:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7aa7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3305:x1876699519260800:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760784.764492:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7aa7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3305:x1876699519260800:12345-0@lo:400:kworker.0 Request processed in 39us (972us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760784.764499:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 524 00000100:00100000:0.0:1789760784.764513:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 525 00000100:00100000:0.0:1789760784.764519:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876699519261056:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760784.764551:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876699519261056:12345-0@lo:400:kworker.0 Request processed in 33us (328us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760784.764557:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 525 00000100:00100000:0.0:1789760784.764569:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 526 00000100:00100000:0.0:1789760784.764576:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876699519261184:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760784.764603:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876699519261184:12345-0@lo:400:kworker.0 Request processed in 27us (267us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760784.764609:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 526 00000100:00100000:0.0:1789760784.764639:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc2886680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3303:1876699519261184:0@lo:400:kworker.0 00000100:00100000:0.0:1789760784.764649:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc2884a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3303:1876699519261056:0@lo:400:kworker.0 00000100:00080000:1.0:1789760784.764896:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760784.764922:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc2885f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3305:1876699519260800:0@lo:400:kworker.0 00000100:00100000:3.0:1789760784.765739:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493847168 00000100:00100000:3.0:1789760784.765749:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493847168 00000100:00100000:3.0:1789760784.765813:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795024029378 time 795070534404 state 0x0 ref 1 00000100:00100000:3.0:1789760784.765833:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795070794820 time 795070534404 state 0x1 ref 1 00000100:00100000:3.0:1789760784.765991:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795070794820 time 795107024976 state 0x0 ref 1 00000100:00100000:3.0:1789760784.766007:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493847168:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760784.766069:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27566080, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:3.0:1789760784.766080:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06e30 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.766100:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=795s v=(0 0) 00010000:00010000:3.0:1789760784.766105:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.766112:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06e30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.766129:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06e30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.766770:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519260672 00000100:00100000:3.0:1789760784.766778:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519260672 00000100:00100000:3.0:1789760784.766831:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 416 00000100:00100000:3.0:1789760784.766840:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d06850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3303:x1876699519260672:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760784.766941:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d06850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3303:x1876699519260672:12345-0@lo:400:kworker.0 Request processed in 102us (2654us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760784.766949:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 416 00000100:00100000:3.0:1789760784.767046:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc2885c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3303:1876699519260672:0@lo:400:kworker.0 00000001:00080000:1.0:1789760784.804707:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974049 is committed 00002000:00010000:2.0:1789760784.804856:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27566080 -> 27570176 00010000:00010000:2.0:1789760784.804878:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06e30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.804893:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06e30 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.804903:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06e30 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.804911:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760784.804916:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06e30 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.804925:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06e30 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.804950:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06e30 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.805036:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e06e30 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27566080->27570175] (req 27566080->27570175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.805261:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493847168:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39248us (42134us total) trans 4294974049 rc 0/0 00000100:00100000:2.0:1789760784.805274:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6760 00000100:00100000:2.0:1789760784.805286:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795070794820 time 795107024976 state 0x2 ref 0 00000100:00100000:2.0:1789760784.809633:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760784.809745:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493847296 00000100:00100000:3.0:1789760784.809760:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493847296 00000100:00100000:3.0:1789760784.809833:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795070794820 time 795107024976 state 0x0 ref 1 00000100:00100000:3.0:1789760784.809850:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795107285392 time 795107024976 state 0x1 ref 1 00000100:00100000:3.0:1789760784.809884:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795107285392 time 795150919660 state 0x0 ref 1 00000100:00100000:3.0:1789760784.809899:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493847296:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760784.810015:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27570176, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:3.0:1789760784.810027:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06e37 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.810046:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=795s v=(0 0) 00010000:00010000:3.0:1789760784.810052:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.810059:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06e37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.810078:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06e37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.842842:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974050 is committed 00002000:00010000:1.0:1789760784.842999:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27570176 -> 27574272 00010000:00010000:1.0:1789760784.843020:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06e37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.843036:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06e37 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.843047:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06e37 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.843057:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760784.843063:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06e37 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.843074:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06e37 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.843100:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06e37 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.843108:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e06e37 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27570176->27574271] (req 27570176->27574271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.843399:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493847296:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33494us (33773us total) trans 4294974050 rc 0/0 00000100:00100000:1.0:1789760784.843413:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6761 00000100:00100000:1.0:1789760784.843426:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795107285392 time 795150919660 state 0x2 ref 0 00000100:00100000:1.0:1789760784.849085:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760784.849166:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493847424 00000100:00100000:0.0:1789760784.849179:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493847424 00000100:00100000:0.0:1789760784.849307:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795107285392 time 795150919660 state 0x0 ref 1 00000100:00100000:0.0:1789760784.849327:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795151180076 time 795150919660 state 0x1 ref 1 00000100:00100000:0.0:1789760784.849363:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795151180076 time 795190398014 state 0x0 ref 1 00000100:00100000:0.0:1789760784.849376:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a37480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493847424:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760784.849555:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27574272, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:0.0:1789760784.849566:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06e3e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.849587:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=795s v=(0 0) 00010000:00010000:0.0:1789760784.849592:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.849667:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06e3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.849687:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06e3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.896029:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974051 is committed 00002000:00010000:2.0:1789760784.896289:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27574272 -> 27578368 00010000:00010000:2.0:1789760784.896309:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06e3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.896326:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06e3e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.896336:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06e3e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.896344:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760784.896350:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06e3e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.896360:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06e3e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.896385:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06e3e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.896394:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e06e3e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27574272->27578367] (req 27574272->27578367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.896628:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a37480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493847424:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47246us (47549us total) trans 4294974051 rc 0/0 00000100:00100000:2.0:1789760784.896643:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6762 00000100:00100000:2.0:1789760784.896656:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795151180076 time 795190398014 state 0x2 ref 0 00000100:00100000:2.0:1789760784.901201:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760784.901404:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493847552 00000100:00100000:0.0:1789760784.901418:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493847552 00000100:00100000:0.0:1789760784.901564:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795151180076 time 795190398014 state 0x0 ref 1 00000100:00100000:0.0:1789760784.901585:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795190658430 time 795190398014 state 0x1 ref 1 00000100:00100000:0.0:1789760784.901620:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795190658430 time 795242654811 state 0x0 ref 1 00000100:00100000:0.0:1789760784.901749:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493847552:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760784.901821:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27578368, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:0.0:1789760784.901832:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06e45 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.901852:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=795s v=(0 0) 00010000:00010000:0.0:1789760784.901858:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.901866:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06e45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.901883:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06e45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.954451:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760784.967607:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974052 is committed 00002000:00010000:1.0:1789760784.967720:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27578368 -> 27582464 00010000:00010000:1.0:1789760784.967742:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06e45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.967760:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06e45 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.967776:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06e45 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.967785:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760784.967791:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06e45 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.967802:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06e45 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.967831:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06e45 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.967839:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06e45 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27578368->27582463] (req 27578368->27582463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.968059:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493847552:12345-192.168.203.60@tcp:4:dd.0 Request processed in 66417us (66865us total) trans 4294974052 rc 0/0 00000100:00100000:1.0:1789760784.968074:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6763 00000100:00100000:1.0:1789760784.968089:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795190658430 time 795242654811 state 0x2 ref 0 00000100:00100000:3.0:1789760784.971047:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760784.971135:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493847680 00000100:00100000:0.0:1789760784.971150:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493847680 00000100:00100000:0.0:1789760784.971242:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795190658430 time 795242654811 state 0x0 ref 1 00000100:00100000:0.0:1789760784.971264:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795242915227 time 795242654811 state 0x1 ref 1 00000100:00100000:0.0:1789760784.972060:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795242915227 time 795313087955 state 0x0 ref 1 00000100:00100000:0.0:1789760784.972082:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493847680:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760784.972151:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27582464, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:0.0:1789760784.972172:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06e4c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.972199:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=795s v=(0 0) 00010000:00010000:0.0:1789760784.972205:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.972212:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06e4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.972229:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06e4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.026141:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974053 is committed 00002000:00010000:2.0:1789760785.026429:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27582464 -> 27586560 00010000:00010000:2.0:1789760785.026449:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06e4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.026466:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06e4c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.026475:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06e4c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.026483:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760785.026488:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06e4c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.026498:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06e4c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.026523:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06e4c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.026530:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e06e4c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27582464->27586559] (req 27582464->27586559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.026654:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493847680:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54571us (55618us total) trans 4294974053 rc 0/0 00000100:00100000:2.0:1789760785.026666:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6764 00000100:00100000:2.0:1789760785.026680:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795242915227 time 795313087955 state 0x2 ref 0 00000100:00100000:2.0:1789760785.029402:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760785.029484:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493847808 00000100:00100000:1.0:1789760785.029498:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493847808 00000100:00080000:2.0:1789760785.029595:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760785.029598:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795242915227 time 795313087955 state 0x0 ref 1 00000100:00100000:1.0:1789760785.029622:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795313348371 time 795313087955 state 0x1 ref 1 00000100:00100000:1.0:1789760785.029663:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795313348371 time 795370697044 state 0x0 ref 1 00000100:00100000:1.0:1789760785.029679:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb452a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493847808:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760785.029746:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27586560, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:1.0:1789760785.029757:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06e53 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.029775:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=795s v=(0 0) 00010000:00010000:1.0:1789760785.029780:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.029788:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06e53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.029804:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06e53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.073014:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974054 is committed 00002000:00010000:0.0:1789760785.073067:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27586560 -> 27590656 00010000:00010000:0.0:1789760785.073088:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06e53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.073103:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06e53 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.073112:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06e53 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.073121:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760785.073127:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06e53 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.073138:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06e53 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.073172:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06e53 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.073181:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e06e53 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27586560->27590655] (req 27586560->27590655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.073321:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb452a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493847808:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43640us (43928us total) trans 4294974054 rc 0/0 00000100:00100000:0.0:1789760785.073337:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6765 00000100:00100000:0.0:1789760785.073349:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795313348371 time 795370697044 state 0x2 ref 0 00000100:00100000:0.0:1789760785.079391:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760785.079517:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493847936 00000100:00100000:2.0:1789760785.079530:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493847936 00000100:00100000:2.0:1789760785.079608:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795313348371 time 795370697044 state 0x0 ref 1 00000100:00100000:2.0:1789760785.079627:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795370957460 time 795370697044 state 0x1 ref 1 00000100:00100000:2.0:1789760785.079662:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795370957460 time 795420696950 state 0x0 ref 1 00000100:00100000:2.0:1789760785.079676:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493847936:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760785.079742:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27590656, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:2.0:1789760785.079754:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06e5a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.079775:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=795s v=(0 0) 00010000:00010000:2.0:1789760785.079781:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.079789:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06e5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.079807:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06e5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.117008:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974055 is committed 00002000:00010000:0.0:1789760785.117173:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27590656 -> 27594752 00010000:00010000:0.0:1789760785.117194:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06e5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.117219:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06e5a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.117229:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06e5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.117238:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760785.117244:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06e5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.117255:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06e5a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.117279:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06e5a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.117288:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06e5a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27590656->27594751] (req 27590656->27594751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.117418:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493847936:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37737us (38038us total) trans 4294974055 rc 0/0 00000100:00100000:0.0:1789760785.117430:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6766 00000100:00100000:0.0:1789760785.117442:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795370957460 time 795420696950 state 0x2 ref 0 00000100:00100000:3.0:1789760785.120049:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760785.120241:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493848064 00000100:00100000:0.0:1789760785.120253:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493848064 00000100:00100000:0.0:1789760785.120327:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795370957460 time 795420696950 state 0x0 ref 1 00000100:00100000:0.0:1789760785.120346:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795420957366 time 795420696950 state 0x1 ref 1 00000100:00100000:0.0:1789760785.120376:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795420957366 time 795461411180 state 0x0 ref 1 00000100:00100000:0.0:1789760785.120389:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493848064:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760785.120457:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27594752, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:0.0:1789760785.120468:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06e61 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.120488:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=795s v=(0 0) 00010000:00010000:0.0:1789760785.120493:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.120501:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06e61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.120517:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06e61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.166298:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974056 is committed 00002000:00010000:0.0:1789760785.166447:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27594752 -> 27598848 00010000:00010000:0.0:1789760785.166468:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06e61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.166484:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06e61 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.166495:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06e61 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.166503:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760785.166508:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06e61 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.166518:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06e61 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.166543:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06e61 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.166551:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06e61 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27594752->27598847] (req 27594752->27598847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.166797:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493848064:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46401us (46752us total) trans 4294974056 rc 0/0 00000100:00100000:0.0:1789760785.166816:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6767 00000100:00100000:0.0:1789760785.166829:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795420957366 time 795461411180 state 0x2 ref 0 00000100:00100000:2.0:1789760785.170380:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760785.170493:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493848192 00000100:00100000:0.0:1789760785.170507:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493848192 00000100:00100000:0.0:1789760785.170584:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795420957366 time 795461411180 state 0x0 ref 1 00000100:00100000:0.0:1789760785.170603:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795461671596 time 795461411180 state 0x1 ref 1 00000100:00100000:0.0:1789760785.170738:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795461671596 time 795511771617 state 0x0 ref 1 00000100:00100000:0.0:1789760785.170755:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb453800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493848192:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760785.170829:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27598848, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:0.0:1789760785.170841:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06e68 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.170862:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=795s v=(0 0) 00010000:00010000:0.0:1789760785.170868:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.170877:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06e68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.170897:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06e68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.212048:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974057 is committed 00002000:00010000:2.0:1789760785.212320:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27598848 -> 27602944 00010000:00010000:2.0:1789760785.212342:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06e68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.212357:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06e68 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.212367:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06e68 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.212375:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760785.212381:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06e68 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.212391:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06e68 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.212416:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06e68 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.212425:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06e68 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27598848->27602943] (req 27598848->27602943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.212579:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb453800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493848192:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41821us (42206us total) trans 4294974057 rc 0/0 00000100:00100000:2.0:1789760785.212593:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6768 00000100:00100000:2.0:1789760785.212607:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795461671596 time 795511771617 state 0x2 ref 0 00000100:00100000:2.0:1789760785.215617:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760785.215708:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493848320 00000100:00100000:0.0:1789760785.215721:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493848320 00000100:00100000:0.0:1789760785.215799:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795461671596 time 795511771617 state 0x0 ref 1 00000100:00100000:0.0:1789760785.215819:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795512032033 time 795511771617 state 0x1 ref 1 00000100:00100000:0.0:1789760785.215856:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795512032033 time 795556890679 state 0x0 ref 1 00000100:00100000:0.0:1789760785.215871:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493848320:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760785.215943:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27602944, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:0.0:1789760785.215956:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06e6f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.216004:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=795s v=(0 0) 00010000:00010000:0.0:1789760785.216009:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.216019:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06e6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.216039:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06e6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.252846:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974058 is committed 00002000:00010000:2.0:1789760785.253063:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27602944 -> 27607040 00010000:00010000:2.0:1789760785.253082:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06e6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.253098:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06e6f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.253107:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06e6f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.253116:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760785.253122:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06e6f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.253133:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06e6f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.253271:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06e6f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.253282:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e06e6f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27602944->27607039] (req 27602944->27607039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.253445:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493848320:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37569us (37836us total) trans 4294974058 rc 0/0 00000100:00100000:2.0:1789760785.253462:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6769 00000100:00100000:2.0:1789760785.253474:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795512032033 time 795556890679 state 0x2 ref 0 00000100:00100000:3.0:1789760785.257316:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760785.257423:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493848448 00000100:00100000:2.0:1789760785.257437:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493848448 00000100:00100000:2.0:1789760785.257511:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795512032033 time 795556890679 state 0x0 ref 1 00000100:00100000:2.0:1789760785.257530:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795557151095 time 795556890679 state 0x1 ref 1 00000100:00100000:2.0:1789760785.257615:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795557151095 time 795598649253 state 0x0 ref 1 00000100:00100000:2.0:1789760785.257640:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493848448:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760785.257709:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27607040, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:2.0:1789760785.257721:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e76 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.257742:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=795s v=(0 0) 00010000:00010000:2.0:1789760785.257747:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.257755:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.257771:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.274149:0:3306:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff962101a34e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519261440:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760785.274275:0:3306:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760785.274404:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519261440 00000100:00100000:3.0:1789760785.274418:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519261440 00000100:00100000:3.0:1789760785.274483:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 217 00000100:00100000:3.0:1789760785.274494:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d8b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519261440:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760785.274541:0:6690:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760785.275297:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d8b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3306:x1876699519261440:12345-0@lo:13:osp-pre-1-0.0 Request processed in 799us (1016us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760785.275309:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 217 00000100:00100000:2.0:1789760785.275449:0:3306:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff962101a34e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3306:1876699519261440:0@lo:13:osp-pre-1-0.0 00000001:00080000:1.0:1789760785.297309:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974059 is committed 00002000:00010000:2.0:1789760785.297417:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27607040 -> 27611136 00010000:00010000:2.0:1789760785.297437:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.297451:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e76 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.297461:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e76 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.297470:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760785.297475:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e76 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.297486:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e76 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.297509:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e76 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.297518:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e76 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27607040->27611135] (req 27607040->27611135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.299069:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493848448:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41426us (41857us total) trans 4294974059 rc 0/0 00000100:00100000:2.0:1789760785.299087:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6770 00000100:00100000:2.0:1789760785.299099:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795557151095 time 795598649253 state 0x2 ref 0 00000100:00100000:1.0:1789760785.302168:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760785.302369:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493848576 00000100:00100000:3.0:1789760785.302383:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493848576 00000100:00100000:3.0:1789760785.302459:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795557151095 time 795598649253 state 0x0 ref 1 00000100:00100000:3.0:1789760785.302477:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795598909669 time 795598649253 state 0x1 ref 1 00000100:00100000:3.0:1789760785.302546:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795598909669 time 795643580030 state 0x0 ref 1 00000100:00100000:3.0:1789760785.302567:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493848576:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760785.302694:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27611136, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:3.0:1789760785.302705:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06e7d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.302725:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=795s v=(0 0) 00010000:00010000:3.0:1789760785.302731:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.302739:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06e7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.302756:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06e7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.341493:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974060 is committed 00002000:00010000:2.0:1789760785.341620:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27611136 -> 27615232 00010000:00010000:2.0:1789760785.341734:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06e7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.341749:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06e7d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.341758:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06e7d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.341767:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760785.341773:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06e7d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.341784:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06e7d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.341808:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06e7d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.341816:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e06e7d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27611136->27615231] (req 27611136->27615231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.341946:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493848576:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39374us (39795us total) trans 4294974060 rc 0/0 00000100:00100000:2.0:1789760785.342004:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6771 00000100:00100000:2.0:1789760785.342018:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795598909669 time 795643580030 state 0x2 ref 0 00000100:00100000:1.0:1789760785.345192:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760785.345300:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493848704 00000100:00100000:2.0:1789760785.345312:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493848704 00000100:00100000:2.0:1789760785.345386:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795598909669 time 795643580030 state 0x0 ref 1 00000100:00100000:2.0:1789760785.345405:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795643840446 time 795643580030 state 0x1 ref 1 00000100:00100000:2.0:1789760785.345446:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795643840446 time 795686480598 state 0x0 ref 1 00000100:00100000:2.0:1789760785.345460:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493848704:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760785.345524:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27615232, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:2.0:1789760785.345535:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e84 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.345554:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=795s v=(0 0) 00010000:00010000:2.0:1789760785.345559:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.345566:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.345594:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.380055:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974061 is committed 00002000:00010000:0.0:1789760785.380267:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27615232 -> 27619328 00010000:00010000:0.0:1789760785.380288:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.380302:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e84 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.380312:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e84 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.380322:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760785.380327:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e84 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.380336:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e84 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.380360:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e84 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.380368:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06e84 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27615232->27619327] (req 27615232->27619327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.380538:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493848704:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35073us (35352us total) trans 4294974061 rc 0/0 00000100:00100000:0.0:1789760785.380551:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6772 00000100:00100000:0.0:1789760785.380563:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795643840446 time 795686480598 state 0x2 ref 0 00000100:00100000:2.0:1789760785.384739:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760785.384860:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493848832 00000100:00100000:0.0:1789760785.384875:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493848832 00000100:00100000:0.0:1789760785.384952:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795643840446 time 795686480598 state 0x0 ref 1 00000100:00100000:0.0:1789760785.387263:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795686741014 time 795686480598 state 0x1 ref 1 00000100:00100000:3.0:1789760785.387453:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795686741014 time 795728484090 state 0x0 ref 1 00000100:00100000:3.0:1789760785.387488:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493848832:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760785.387563:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27619328, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:3.0:1789760785.387575:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06e8b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.387596:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=795s v=(0 0) 00010000:00010000:3.0:1789760785.387602:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.387610:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06e8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.387723:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06e8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.418188:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974062 is committed 00002000:00010000:2.0:1789760785.418497:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27619328 -> 27623424 00010000:00010000:2.0:1789760785.418518:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06e8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.418533:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06e8b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.418543:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06e8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.418551:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760785.418557:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06e8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.418566:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06e8b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.418591:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06e8b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.418599:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e06e8b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27619328->27623423] (req 27619328->27623423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.418791:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493848832:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31315us (34061us total) trans 4294974062 rc 0/0 00000100:00100000:2.0:1789760785.418806:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6773 00000100:00100000:2.0:1789760785.418819:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795686741014 time 795728484090 state 0x2 ref 0 00000100:00100000:2.0:1789760785.421574:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760785.421799:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493848960 00000100:00100000:1.0:1789760785.421814:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493848960 00000100:00100000:1.0:1789760785.421897:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795686741014 time 795728484090 state 0x0 ref 1 00000100:00100000:1.0:1789760785.421915:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795728744506 time 795728484090 state 0x1 ref 1 00000100:00100000:1.0:1789760785.421954:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795728744506 time 795762989027 state 0x0 ref 1 00000100:00100000:1.0:1789760785.421995:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493848960:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760785.422067:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27623424, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:1.0:1789760785.422080:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06e92 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.422102:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=795s v=(0 0) 00010000:00010000:1.0:1789760785.422107:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.422115:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06e92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.422132:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06e92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760785.457863:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27623424 -> 27627520 00010000:00010000:0.0:1789760785.457891:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06e92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.457907:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06e92 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.457917:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06e92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.457925:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760785.457931:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06e92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.457942:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06e92 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.457993:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06e92 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.458001:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06e92 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27623424->27627519] (req 27623424->27627519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.458128:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493848960:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36128us (36562us total) trans 4294974063 rc 0/0 00000100:00100000:0.0:1789760785.458141:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6774 00000100:00100000:0.0:1789760785.458164:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795728744506 time 795762989027 state 0x2 ref 0 00000001:00080000:2.0:1789760785.461125:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974063 is committed 00000100:00100000:1.0:1789760785.462814:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760785.462929:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493849088 00000100:00100000:3.0:1789760785.462942:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493849088 00000100:00100000:3.0:1789760785.463136:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795728744506 time 795762989027 state 0x0 ref 1 00000100:00100000:3.0:1789760785.463165:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795763249443 time 795762989027 state 0x1 ref 1 00000100:00100000:3.0:1789760785.463246:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795763249443 time 795804279787 state 0x0 ref 1 00000100:00100000:3.0:1789760785.463261:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493849088:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760785.463325:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27627520, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:3.0:1789760785.463335:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06e99 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.463357:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=795s v=(0 0) 00010000:00010000:3.0:1789760785.463362:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.463370:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06e99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.463387:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06e99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760785.495137:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27627520 -> 27631616 00000001:00080000:1.0:1789760785.495167:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974064 is committed 00010000:00010000:2.0:1789760785.495178:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06e99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.495196:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06e99 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.495208:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06e99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.495216:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760785.495222:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06e99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.495232:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06e99 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.495257:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06e99 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.495266:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06e99 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27627520->27631615] (req 27627520->27631615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.495603:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493849088:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32337us (32796us total) trans 4294974064 rc 0/0 00000100:00100000:2.0:1789760785.495615:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6775 00000100:00100000:2.0:1789760785.495627:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795763249443 time 795804279787 state 0x2 ref 0 00000100:00100000:2.0:1789760785.498570:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760785.499034:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493849216 00000100:00100000:0.0:1789760785.499049:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493849216 00000100:00100000:0.0:1789760785.499131:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795763249443 time 795804279787 state 0x0 ref 1 00000100:00100000:0.0:1789760785.499161:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795804540203 time 795804279787 state 0x1 ref 1 00000100:00100000:1.0:1789760785.499290:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795804540203 time 795840321729 state 0x0 ref 1 00000100:00100000:1.0:1789760785.499308:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493849216:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760785.499370:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27631616, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:1.0:1789760785.499381:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ea0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.499401:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=795s v=(0 0) 00010000:00010000:1.0:1789760785.499407:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.499414:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ea0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.499431:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ea0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760785.535780:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27631616 -> 27635712 00010000:00010000:0.0:1789760785.535809:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ea0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.535825:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ea0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.535834:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ea0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.535843:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760785.535848:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ea0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.535859:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ea0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.535885:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ea0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.535894:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06ea0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27631616->27635711] (req 27631616->27635711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.536059:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493849216:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36748us (37496us total) trans 4294974065 rc 0/0 00000100:00100000:0.0:1789760785.536073:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6776 00000100:00100000:0.0:1789760785.536163:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795804540203 time 795840321729 state 0x2 ref 0 00000001:00080000:1.0:1789760785.536419:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974065 is committed 00000100:00100000:1.0:1789760785.540950:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760785.541178:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493849344 00000100:00100000:2.0:1789760785.541192:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493849344 00000100:00100000:2.0:1789760785.541340:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795804540203 time 795840321729 state 0x0 ref 1 00000100:00100000:2.0:1789760785.541359:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795840582145 time 795840321729 state 0x1 ref 1 00000100:00100000:2.0:1789760785.541399:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795840582145 time 795882434034 state 0x0 ref 1 00000100:00100000:2.0:1789760785.541415:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a34e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493849344:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760785.541486:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27635712, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:2.0:1789760785.541498:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06ea7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.541519:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=795s v=(0 0) 00010000:00010000:2.0:1789760785.541525:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.541533:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06ea7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.541550:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06ea7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.579440:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974066 is committed 00002000:00010000:0.0:1789760785.579585:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27635712 -> 27639808 00010000:00010000:0.0:1789760785.579605:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06ea7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.579622:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06ea7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.579644:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06ea7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.579653:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760785.579659:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06ea7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.579670:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06ea7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.579697:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06ea7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.579712:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e06ea7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27635712->27639807] (req 27635712->27639807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.579857:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a34e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493849344:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38437us (38915us total) trans 4294974066 rc 0/0 00000100:00100000:0.0:1789760785.579871:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6777 00000100:00100000:0.0:1789760785.579884:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795840582145 time 795882434034 state 0x2 ref 0 00000100:00100000:1.0:1789760785.584212:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760785.584406:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493849472 00000100:00100000:0.0:1789760785.584418:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493849472 00000100:00100000:0.0:1789760785.584491:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795840582145 time 795882434034 state 0x0 ref 1 00000100:00100000:0.0:1789760785.584510:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795882694450 time 795882434034 state 0x1 ref 1 00000100:00100000:0.0:1789760785.584537:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795882694450 time 795925572112 state 0x0 ref 1 00000100:00100000:0.0:1789760785.584551:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493849472:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760785.584621:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27639808, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xd809 00010000:00010000:0.0:1789760785.584641:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e06eae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.584663:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=795s v=(0 0) 00010000:00010000:0.0:1789760785.584668:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.584676:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e06eae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.584700:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e06eae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.649346:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974067 is committed 00002000:00010000:0.0:1789760785.649569:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27639808 -> 27643904 00002000:00010000:0.0:1789760785.649575:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760784 -> 1789760785 00002000:00010000:0.0:1789760785.649579:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760784 -> 1789760785 00010000:00010000:0.0:1789760785.649598:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e06eae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.649615:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e06eae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.649626:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e06eae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.649703:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760785.649709:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e06eae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.649720:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e06eae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.649747:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e06eae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.649763:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e06eae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27639808->27643903] (req 27639808->27643903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.649941:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493849472:12345-192.168.203.60@tcp:4:dd.0 Request processed in 65383us (65748us total) trans 4294974067 rc 0/0 00000100:00100000:0.0:1789760785.649955:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6778 00000100:00100000:0.0:1789760785.650017:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795882694450 time 795925572112 state 0x2 ref 0 00000100:00100000:1.0:1789760785.653872:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760785.654271:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493849600 00000100:00100000:3.0:1789760785.654286:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493849600 00000100:00100000:3.0:1789760785.654359:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795882694450 time 795925572112 state 0x0 ref 1 00000100:00100000:3.0:1789760785.654378:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795925832528 time 795925572112 state 0x1 ref 1 00000100:00100000:3.0:1789760785.654622:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795925832528 time 795995654779 state 0x0 ref 1 00000100:00100000:3.0:1789760785.654711:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493849600:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760785.654784:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27643904, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:3.0:1789760785.654796:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06eb5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.654818:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=795s v=(0 0) 00010000:00010000:3.0:1789760785.654823:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.654831:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06eb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.654849:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06eb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.697511:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974068 is committed 00002000:00010000:3.0:1789760785.697706:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27643904 -> 27648000 00010000:00010000:3.0:1789760785.697730:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06eb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.697754:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06eb5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.697764:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06eb5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.697772:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760785.697778:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06eb5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.697870:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06eb5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.697904:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06eb5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.697914:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06eb5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27643904->27647999] (req 27643904->27647999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.698512:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493849600:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43796us (44644us total) trans 4294974068 rc 0/0 00000100:00100000:3.0:1789760785.698528:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6779 00000100:00100000:3.0:1789760785.698541:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795925832528 time 795995654779 state 0x2 ref 0 00000100:00100000:1.0:1789760785.702283:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760785.702468:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493849728 00000100:00100000:0.0:1789760785.702483:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493849728 00000100:00100000:0.0:1789760785.702561:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795925832528 time 795995654779 state 0x0 ref 1 00000100:00100000:0.0:1789760785.702580:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795995915195 time 795995654779 state 0x1 ref 1 00000100:00100000:0.0:1789760785.702615:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795995915195 time 796043650091 state 0x0 ref 1 00000100:00100000:0.0:1789760785.702629:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493849728:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760785.702699:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27648000, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:0.0:1789760785.702710:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06ebc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.702730:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=796s v=(0 0) 00010000:00010000:0.0:1789760785.702735:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.702743:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06ebc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.702765:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06ebc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.766891:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974069 is committed 00002000:00010000:2.0:1789760785.767215:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27648000 -> 27652096 00010000:00010000:2.0:1789760785.767236:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06ebc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.767252:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06ebc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.767263:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06ebc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.767270:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760785.767276:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06ebc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.767286:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06ebc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.767310:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06ebc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.767318:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06ebc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27648000->27652095] (req 27648000->27652095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.767436:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493849728:12345-192.168.203.60@tcp:4:dd.0 Request processed in 64801us (65218us total) trans 4294974069 rc 0/0 00000100:00100000:2.0:1789760785.767451:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6780 00000100:00100000:2.0:1789760785.767463:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795995915195 time 796043650091 state 0x2 ref 0 00000100:00100000:0.0:1789760785.774447:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760785.774899:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493849856 00000100:00100000:2.0:1789760785.774912:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493849856 00000100:00100000:2.0:1789760785.775037:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 795995915195 time 796043650091 state 0x0 ref 1 00000100:00100000:2.0:1789760785.775058:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796043910507 time 796043650091 state 0x1 ref 1 00000100:00100000:2.0:1789760785.775169:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796043910507 time 796116202590 state 0x0 ref 1 00000100:00100000:2.0:1789760785.775186:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493849856:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760785.775252:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27652096, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:2.0:1789760785.775264:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06ec3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.775285:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=796s v=(0 0) 00010000:00010000:2.0:1789760785.775290:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.775297:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06ec3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.775313:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06ec3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.844625:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974070 is committed 00002000:00010000:1.0:1789760785.844824:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27652096 -> 27656192 00010000:00010000:1.0:1789760785.844844:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06ec3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.844859:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06ec3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.844868:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06ec3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.844877:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760785.844883:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06ec3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.844900:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06ec3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.844924:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06ec3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.844933:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06ec3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27652096->27656191] (req 27652096->27656191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.845324:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493849856:12345-192.168.203.60@tcp:4:dd.0 Request processed in 70132us (70884us total) trans 4294974070 rc 0/0 00000100:00100000:1.0:1789760785.845338:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6781 00000100:00100000:1.0:1789760785.845352:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796043910507 time 796116202590 state 0x2 ref 0 00000100:00100000:1.0:1789760785.848656:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760785.848764:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493849984 00000100:00100000:2.0:1789760785.848778:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493849984 00000100:00100000:2.0:1789760785.848856:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796043910507 time 796116202590 state 0x0 ref 1 00000100:00100000:2.0:1789760785.848876:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796116463006 time 796116202590 state 0x1 ref 1 00000100:00100000:2.0:1789760785.848916:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796116463006 time 796189951032 state 0x0 ref 1 00000100:00100000:2.0:1789760785.848931:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493849984:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760785.849042:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27656192, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:2.0:1789760785.849053:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06eca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.849082:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=796s v=(0 0) 00010000:00010000:2.0:1789760785.849088:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.849096:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06eca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.849113:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06eca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760785.898072:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27656192 -> 27660288 00010000:00010000:2.0:1789760785.898104:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06eca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.898122:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06eca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.898132:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06eca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.898142:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760785.898158:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06eca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.898169:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06eca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.898194:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06eca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.898211:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06eca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27656192->27660287] (req 27656192->27660287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.898334:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493849984:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49397us (49696us total) trans 4294974071 rc 0/0 00000100:00100000:2.0:1789760785.898347:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6782 00000100:00100000:2.0:1789760785.898361:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796116463006 time 796189951032 state 0x2 ref 0 00000001:00080000:0.0:1789760785.898521:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974071 is committed 00000100:00100000:1.0:1789760785.902100:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760785.902317:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493850112 00000100:00100000:2.0:1789760785.902330:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493850112 00000100:00100000:2.0:1789760785.902418:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796116463006 time 796189951032 state 0x0 ref 1 00000100:00100000:2.0:1789760785.902438:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796190211448 time 796189951032 state 0x1 ref 1 00000100:00100000:2.0:1789760785.902551:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796190211448 time 796243584910 state 0x0 ref 1 00000100:00100000:2.0:1789760785.902571:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493850112:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760785.902650:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27660288, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:2.0:1789760785.902662:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06ed1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.902682:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=796s v=(0 0) 00010000:00010000:2.0:1789760785.902688:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.902696:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06ed1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.902713:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06ed1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.950126:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974072 is committed 00002000:00010000:0.0:1789760785.950360:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27660288 -> 27664384 00010000:00010000:0.0:1789760785.950381:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06ed1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.950395:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06ed1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.950405:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06ed1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.950412:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760785.950417:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06ed1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.950427:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06ed1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.950451:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06ed1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.950465:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e06ed1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27660288->27664383] (req 27660288->27664383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.950707:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493850112:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48133us (48615us total) trans 4294974072 rc 0/0 00000100:00100000:0.0:1789760785.950720:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6783 00000100:00100000:0.0:1789760785.950733:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796190211448 time 796243584910 state 0x2 ref 0 00000100:00100000:1.0:1789760785.954353:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760785.954448:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493850240 00000100:00100000:0.0:1789760785.954462:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493850240 00000100:00100000:0.0:1789760785.954537:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796190211448 time 796243584910 state 0x0 ref 1 00000100:00100000:0.0:1789760785.954555:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796243845326 time 796243584910 state 0x1 ref 1 00000100:00100000:0.0:1789760785.954591:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796243845326 time 796295626645 state 0x0 ref 1 00000100:00100000:0.0:1789760785.954606:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493850240:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760785.954670:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27664384, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:0.0:1789760785.954681:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06ed8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.954703:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=796s v=(0 0) 00010000:00010000:0.0:1789760785.954708:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.954716:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06ed8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.954738:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06ed8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.978554:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760785.997176:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974073 is committed 00002000:00010000:0.0:1789760785.997337:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27664384 -> 27668480 00010000:00010000:0.0:1789760785.997357:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06ed8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.997370:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06ed8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.997379:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06ed8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.997388:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760785.997393:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06ed8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.997403:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06ed8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.997427:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06ed8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.997435:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06ed8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27664384->27668479] (req 27664384->27668479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.997549:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493850240:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42939us (43204us total) trans 4294974073 rc 0/0 00000100:00100000:0.0:1789760785.997563:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6784 00000100:00100000:0.0:1789760785.997576:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796243845326 time 796295626645 state 0x2 ref 0 00000100:00100000:2.0:1789760786.010424:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760786.010532:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493850368 00000100:00100000:1.0:1789760786.010546:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493850368 00000100:00100000:1.0:1789760786.010630:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796243845326 time 796295626645 state 0x0 ref 1 00000100:00100000:1.0:1789760786.010649:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796295887061 time 796295626645 state 0x1 ref 1 00000100:00100000:1.0:1789760786.010686:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796295887061 time 796351721128 state 0x0 ref 1 00000100:00100000:1.0:1789760786.010705:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fadb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493850368:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760786.010784:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27668480, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:1.0:1789760786.010797:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06edf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.010820:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=796s v=(0 0) 00010000:00010000:1.0:1789760786.010827:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.010834:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06edf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.010853:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06edf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.061269:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974074 is committed 00002000:00010000:2.0:1789760786.061418:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27668480 -> 27672576 00010000:00010000:2.0:1789760786.061440:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06edf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.061456:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06edf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.061466:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06edf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.061475:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760786.061481:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06edf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.061492:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06edf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.061517:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06edf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.061526:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e06edf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27668480->27672575] (req 27668480->27672575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.061687:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fadb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493850368:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50978us (51271us total) trans 4294974074 rc 0/0 00000100:00100000:2.0:1789760786.061701:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6785 00000100:00100000:2.0:1789760786.061712:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796295887061 time 796351721128 state 0x2 ref 0 00000100:00100000:0.0:1789760786.064846:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760786.064946:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493850496 00000100:00100000:2.0:1789760786.064991:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493850496 00000100:00100000:2.0:1789760786.065074:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796295887061 time 796351721128 state 0x0 ref 1 00000100:00100000:2.0:1789760786.065094:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796351981544 time 796351721128 state 0x1 ref 1 00000100:00100000:2.0:1789760786.065122:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796351981544 time 796406157416 state 0x0 ref 1 00000100:00100000:2.0:1789760786.065138:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493850496:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760786.065218:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27672576, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:2.0:1789760786.065230:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06ee6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.065249:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=796s v=(0 0) 00010000:00010000:2.0:1789760786.065254:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.065262:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06ee6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.065278:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06ee6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.098602:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974075 is committed 00002000:00010000:0.0:1789760786.098809:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27672576 -> 27676672 00010000:00010000:0.0:1789760786.098834:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06ee6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.098849:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06ee6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.098858:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06ee6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.098866:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760786.098871:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06ee6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.098881:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06ee6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.098908:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06ee6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.098922:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06ee6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27672576->27676671] (req 27672576->27676671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.099101:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493850496:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33958us (34261us total) trans 4294974075 rc 0/0 00000100:00100000:0.0:1789760786.099114:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6786 00000100:00100000:0.0:1789760786.099127:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796351981544 time 796406157416 state 0x2 ref 0 00000100:00100000:1.0:1789760786.103393:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760786.103488:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493850624 00000100:00100000:0.0:1789760786.103502:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493850624 00000100:00100000:0.0:1789760786.103577:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796351981544 time 796406157416 state 0x0 ref 1 00000100:00100000:0.0:1789760786.103597:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796406417832 time 796406157416 state 0x1 ref 1 00000100:00100000:0.0:1789760786.103631:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796406417832 time 796444666413 state 0x0 ref 1 00000100:00100000:0.0:1789760786.103645:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4361500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493850624:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760786.103711:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27676672, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:0.0:1789760786.103723:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06eed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.103745:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=796s v=(0 0) 00010000:00010000:0.0:1789760786.103751:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.103759:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06eed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.103783:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06eed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760786.132155:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27676672 -> 27680768 00010000:00010000:0.0:1789760786.132185:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06eed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.132209:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06eed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.132219:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06eed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.132227:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760786.132232:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06eed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.132241:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06eed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.132265:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06eed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.132273:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06eed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27676672->27680767] (req 27676672->27680767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.132396:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4361500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493850624:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28745us (29011us total) trans 4294974076 rc 0/0 00000100:00100000:0.0:1789760786.132408:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6787 00000100:00100000:0.0:1789760786.132420:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796406417832 time 796444666413 state 0x2 ref 0 00000001:00080000:1.0:1789760786.132943:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974076 is committed 00000100:00100000:3.0:1789760786.135052:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760786.135399:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493850752 00000100:00100000:0.0:1789760786.135414:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493850752 00000100:00100000:0.0:1789760786.135491:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796406417832 time 796444666413 state 0x0 ref 1 00000100:00100000:0.0:1789760786.135510:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796444926829 time 796444666413 state 0x1 ref 1 00000100:00100000:0.0:1789760786.135551:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796444926829 time 796476585462 state 0x0 ref 1 00000100:00100000:0.0:1789760786.135567:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493850752:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760786.135633:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27680768, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:0.0:1789760786.135644:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06ef4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.135665:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=796s v=(0 0) 00010000:00010000:0.0:1789760786.135671:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.135678:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06ef4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.135694:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06ef4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760786.168683:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27680768 -> 27684864 00010000:00010000:0.0:1789760786.168712:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06ef4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.168728:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06ef4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.168739:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06ef4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.168747:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760786.168753:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06ef4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.168764:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06ef4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.168790:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06ef4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.168799:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e06ef4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27680768->27684863] (req 27680768->27684863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.168924:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493850752:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33352us (33879us total) trans 4294974077 rc 0/0 00000100:00100000:0.0:1789760786.168941:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6788 00000100:00100000:0.0:1789760786.168954:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796444926829 time 796476585462 state 0x2 ref 0 00000001:00080000:2.0:1789760786.172736:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974077 is committed 00000100:00100000:0.0:1789760786.173850:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760786.174101:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493850880 00000100:00100000:2.0:1789760786.174114:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493850880 00000100:00100000:2.0:1789760786.174210:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796444926829 time 796476585462 state 0x0 ref 1 00000100:00100000:2.0:1789760786.174230:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796476845878 time 796476585462 state 0x1 ref 1 00000100:00100000:2.0:1789760786.174296:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796476845878 time 796515329695 state 0x0 ref 1 00000100:00100000:2.0:1789760786.174311:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493850880:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760786.174378:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27684864, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:2.0:1789760786.174390:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06efb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.174411:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=796s v=(0 0) 00010000:00010000:2.0:1789760786.174416:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.174425:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06efb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.174440:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06efb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.204130:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974078 is committed 00002000:00010000:2.0:1789760786.204303:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27684864 -> 27688960 00010000:00010000:2.0:1789760786.204324:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06efb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.204340:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06efb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.204350:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06efb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.204358:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760786.204363:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06efb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.204372:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06efb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.204396:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06efb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.204404:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06efb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27684864->27688959] (req 27684864->27688959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.204565:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493850880:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30247us (30713us total) trans 4294974078 rc 0/0 00000100:00100000:2.0:1789760786.204580:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6789 00000100:00100000:2.0:1789760786.204595:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796476845878 time 796515329695 state 0x2 ref 0 00000100:00100000:3.0:1789760786.208532:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760786.208719:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493851008 00000100:00100000:2.0:1789760786.208733:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493851008 00000100:00100000:2.0:1789760786.208811:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796476845878 time 796515329695 state 0x0 ref 1 00000100:00100000:2.0:1789760786.208830:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796515590111 time 796515329695 state 0x1 ref 1 00000100:00100000:2.0:1789760786.208919:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796515590111 time 796549953215 state 0x0 ref 1 00000100:00100000:2.0:1789760786.208935:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493851008:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760786.209040:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27688960, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:2.0:1789760786.209052:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06f02 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.209073:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=796s v=(0 0) 00010000:00010000:2.0:1789760786.209078:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.209086:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06f02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.209103:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06f02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.236059:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974079 is committed 00002000:00010000:2.0:1789760786.236291:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27688960 -> 27693056 00010000:00010000:2.0:1789760786.236312:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06f02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.236326:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06f02 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.236336:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06f02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.236343:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760786.236348:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06f02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.236357:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06f02 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.236380:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06f02 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.236387:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06f02 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27688960->27693055] (req 27688960->27693055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.236545:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493851008:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27606us (28025us total) trans 4294974079 rc 0/0 00000100:00100000:2.0:1789760786.236562:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6790 00000100:00100000:2.0:1789760786.236574:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796515590111 time 796549953215 state 0x2 ref 0 00000100:00100000:3.0:1789760786.240107:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760786.240531:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493851136 00000100:00100000:0.0:1789760786.240544:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493851136 00000100:00100000:0.0:1789760786.240622:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796515590111 time 796549953215 state 0x0 ref 1 00000100:00100000:0.0:1789760786.240712:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796550213631 time 796549953215 state 0x1 ref 1 00000100:00100000:0.0:1789760786.240753:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796550213631 time 796581787838 state 0x0 ref 1 00000100:00100000:0.0:1789760786.240769:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493851136:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760786.240838:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27693056, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:0.0:1789760786.240850:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06f09 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.240871:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=796s v=(0 0) 00010000:00010000:0.0:1789760786.240877:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.240886:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06f09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.240903:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06f09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.277647:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974080 is committed 00002000:00010000:1.0:1789760786.278050:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27693056 -> 27697152 00010000:00010000:1.0:1789760786.278073:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06f09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.278090:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06f09 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.278101:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06f09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.278116:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760786.278122:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06f09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.278133:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06f09 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.278171:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06f09 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.278180:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e06f09 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27693056->27697151] (req 27693056->27697151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.278302:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493851136:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37529us (38202us total) trans 4294974080 rc 0/0 00000100:00100000:1.0:1789760786.278313:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6791 00000100:00100000:1.0:1789760786.278326:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796550213631 time 796581787838 state 0x2 ref 0 00000100:00100000:1.0:1789760786.285317:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760786.285779:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493851264 00000100:00100000:0.0:1789760786.285793:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493851264 00000100:00100000:0.0:1789760786.285872:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796550213631 time 796581787838 state 0x0 ref 1 00000100:00100000:0.0:1789760786.285892:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796582048254 time 796581787838 state 0x1 ref 1 00000100:00100000:0.0:1789760786.286020:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796582048254 time 796627054255 state 0x0 ref 1 00000100:00100000:0.0:1789760786.286036:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4363480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493851264:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760786.286104:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27697152, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:0.0:1789760786.286116:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06f10 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.286135:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=796s v=(0 0) 00010000:00010000:0.0:1789760786.286140:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.286158:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06f10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.286175:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06f10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.328008:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974081 is committed 00002000:00010000:0.0:1789760786.328291:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27697152 -> 27701248 00010000:00010000:0.0:1789760786.328313:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06f10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.328336:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06f10 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.328347:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06f10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.328355:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760786.328362:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06f10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.328372:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06f10 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.328398:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06f10 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.328406:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c75f9184/0x3eb3dcc0b2e06f10 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27697152->27701247] (req 27697152->27701247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.328572:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4363480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493851264:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42530us (43374us total) trans 4294974081 rc 0/0 00000100:00100000:0.0:1789760786.328586:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6792 00000100:00100000:0.0:1789760786.328598:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796582048254 time 796627054255 state 0x2 ref 0 00000100:00100000:3.0:1789760786.331336:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760786.331419:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493851392 00000100:00100000:2.0:1789760786.331433:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493851392 00000100:00100000:2.0:1789760786.331508:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796582048254 time 796627054255 state 0x0 ref 1 00000100:00100000:2.0:1789760786.331527:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796627314671 time 796627054255 state 0x1 ref 1 00000100:00100000:2.0:1789760786.331557:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796627314671 time 796672591981 state 0x0 ref 1 00000100:00100000:2.0:1789760786.331570:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493851392:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760786.331632:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27701248, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:2.0:1789760786.331644:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06f17 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.331662:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=796s v=(0 0) 00010000:00010000:2.0:1789760786.331667:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.331674:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06f17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.331693:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06f17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.360990:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974082 is committed 00002000:00010000:1.0:1789760786.361235:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27701248 -> 27705344 00010000:00010000:1.0:1789760786.361256:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06f17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.361272:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06f17 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.361289:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06f17 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.361298:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760786.361304:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06f17 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.361315:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06f17 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.361341:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06f17 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.361349:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e06f17 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27701248->27705343] (req 27701248->27705343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.361519:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493851392:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29942us (30189us total) trans 4294974082 rc 0/0 00000100:00100000:1.0:1789760786.361532:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6793 00000100:00100000:1.0:1789760786.361545:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796627314671 time 796672591981 state 0x2 ref 0 00000100:00100000:1.0:1789760786.365152:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760786.365484:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493851520 00000100:00100000:2.0:1789760786.365499:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493851520 00000100:00100000:2.0:1789760786.365577:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796627314671 time 796672591981 state 0x0 ref 1 00000100:00100000:2.0:1789760786.365598:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796672852397 time 796672591981 state 0x1 ref 1 00000100:00100000:2.0:1789760786.365626:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796672852397 time 796706660712 state 0x0 ref 1 00000100:00100000:2.0:1789760786.365654:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493851520:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760786.365725:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27705344, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:2.0:1789760786.365737:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06f1e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.365757:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=796s v=(0 0) 00010000:00010000:2.0:1789760786.365763:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.365771:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06f1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.365788:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06f1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.408617:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974083 is committed 00002000:00010000:1.0:1789760786.408917:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27705344 -> 27709440 00010000:00010000:1.0:1789760786.408937:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06f1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.408956:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06f1e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.408988:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06f1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.408996:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760786.409001:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06f1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.409011:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06f1e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.409035:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06f1e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.409043:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06f1e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27705344->27709439] (req 27705344->27709439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.409211:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493851520:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43546us (44070us total) trans 4294974083 rc 0/0 00000100:00100000:1.0:1789760786.409224:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6794 00000100:00100000:1.0:1789760786.409237:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796672852397 time 796706660712 state 0x2 ref 0 00000100:00100000:1.0:1789760786.412696:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760786.413306:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493851648 00000100:00100000:2.0:1789760786.413320:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493851648 00000100:00100000:2.0:1789760786.413396:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796672852397 time 796706660712 state 0x0 ref 1 00000100:00100000:2.0:1789760786.413414:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796706921128 time 796706660712 state 0x1 ref 1 00000100:00100000:2.0:1789760786.413444:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796706921128 time 796754479846 state 0x0 ref 1 00000100:00100000:2.0:1789760786.413456:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493851648:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760786.413514:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27709440, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:2.0:1789760786.413523:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f25 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.413541:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=796s v=(0 0) 00010000:00010000:2.0:1789760786.413546:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.413553:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.413568:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.450272:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974084 is committed 00002000:00010000:1.0:1789760786.450475:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27709440 -> 27713536 00010000:00010000:1.0:1789760786.450495:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.450510:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f25 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.450520:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.450528:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760786.450533:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.450543:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f25 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.450568:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f25 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.450577:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f25 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27709440->27713535] (req 27709440->27713535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.450699:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493851648:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37236us (38074us total) trans 4294974084 rc 0/0 00000100:00100000:1.0:1789760786.450711:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6795 00000100:00100000:1.0:1789760786.450723:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796706921128 time 796754479846 state 0x2 ref 0 00000100:00100000:3.0:1789760786.454185:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760786.454299:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493851776 00000100:00100000:0.0:1789760786.454311:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493851776 00000100:00100000:0.0:1789760786.454386:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796706921128 time 796754479846 state 0x0 ref 1 00000100:00100000:0.0:1789760786.454404:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796754740262 time 796754479846 state 0x1 ref 1 00000100:00100000:0.0:1789760786.454439:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796754740262 time 796795474045 state 0x0 ref 1 00000100:00100000:0.0:1789760786.454454:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493851776:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760786.454520:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27713536, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:0.0:1789760786.454530:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06f2c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.454549:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=796s v=(0 0) 00010000:00010000:0.0:1789760786.454554:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.454562:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06f2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.454579:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06f2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.490839:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974085 is committed 00002000:00010000:2.0:1789760786.491131:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27713536 -> 27717632 00010000:00010000:2.0:1789760786.491160:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06f2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.491177:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06f2c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.491187:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06f2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.491204:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760786.491210:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06f2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.491222:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06f2c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.491245:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06f2c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.491253:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e06f2c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27713536->27717631] (req 27713536->27717631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.491375:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493851776:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36915us (37197us total) trans 4294974085 rc 0/0 00000100:00100000:2.0:1789760786.491389:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6796 00000100:00100000:2.0:1789760786.491402:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796754740262 time 796795474045 state 0x2 ref 0 00000100:00100000:0.0:1789760786.494382:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760786.494493:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493851904 00000100:00100000:1.0:1789760786.494506:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493851904 00000100:00100000:1.0:1789760786.494584:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796754740262 time 796795474045 state 0x0 ref 1 00000100:00100000:1.0:1789760786.494604:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796795734461 time 796795474045 state 0x1 ref 1 00000100:00100000:1.0:1789760786.494644:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796795734461 time 796835678122 state 0x0 ref 1 00000100:00100000:1.0:1789760786.494661:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493851904:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760786.494735:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27717632, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:1.0:1789760786.494747:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06f33 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.494768:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=796s v=(0 0) 00010000:00010000:1.0:1789760786.494773:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.494782:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06f33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.494904:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06f33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.527512:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974086 is committed 00002000:00010000:1.0:1789760786.527631:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27717632 -> 27721728 00010000:00010000:1.0:1789760786.527655:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06f33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.527678:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06f33 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.527689:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06f33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.527698:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760786.527704:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06f33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.527714:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06f33 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.527741:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06f33 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.527748:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e06f33 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27717632->27721727] (req 27717632->27721727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.528045:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493851904:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33381us (33671us total) trans 4294974086 rc 0/0 00000100:00100000:1.0:1789760786.528131:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6797 00000100:00100000:1.0:1789760786.528157:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796795734461 time 796835678122 state 0x2 ref 0 00000100:00100000:1.0:1789760786.534950:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760786.536208:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493852032 00000100:00100000:2.0:1789760786.536224:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493852032 00000100:00100000:2.0:1789760786.536310:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796795734461 time 796835678122 state 0x0 ref 1 00000100:00100000:2.0:1789760786.536328:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796835938538 time 796835678122 state 0x1 ref 1 00000100:00100000:2.0:1789760786.536954:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796835938538 time 796877984101 state 0x0 ref 1 00000100:00100000:2.0:1789760786.537226:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc352c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493852032:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760786.537302:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27721728, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:2.0:1789760786.537314:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f3a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.537335:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=796s v=(0 0) 00010000:00010000:2.0:1789760786.537340:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.537348:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.537363:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.603066:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974087 is committed 00002000:00010000:2.0:1789760786.603295:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27721728 -> 27725824 00010000:00010000:2.0:1789760786.603316:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.603331:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f3a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.603339:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.603347:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760786.603352:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.603361:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f3a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.603392:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f3a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.603400:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f3a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27721728->27725823] (req 27721728->27725823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.603682:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493852032:12345-192.168.203.60@tcp:4:dd.0 Request processed in 66306us (68584us total) trans 4294974087 rc 0/0 00000100:00100000:2.0:1789760786.603696:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6798 00000100:00100000:2.0:1789760786.603708:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796835938538 time 796877984101 state 0x2 ref 0 00000100:00100000:3.0:1789760786.608673:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760786.609080:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493852160 00000100:00100000:2.0:1789760786.609094:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493852160 00000100:00100000:2.0:1789760786.609183:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796835938538 time 796877984101 state 0x0 ref 1 00000100:00100000:2.0:1789760786.609212:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796878244517 time 796877984101 state 0x1 ref 1 00000100:00100000:2.0:1789760786.609254:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796878244517 time 796950288914 state 0x0 ref 1 00000100:00100000:2.0:1789760786.609268:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493852160:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760786.609337:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27725824, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xd809 00010000:00010000:2.0:1789760786.609347:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06f41 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.609368:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=796s v=(0 0) 00010000:00010000:2.0:1789760786.609374:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.609382:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06f41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.609399:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06f41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.667527:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974088 is committed 00002000:00010000:2.0:1789760786.668542:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27725824 -> 27729920 00002000:00010000:2.0:1789760786.668557:0:13038:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760785 -> 1789760786 00002000:00010000:2.0:1789760786.668560:0:13038:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760785 -> 1789760786 00010000:00010000:2.0:1789760786.668579:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06f41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.668600:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06f41 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.668610:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06f41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.669006:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760786.669013:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06f41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.669026:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06f41 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.669053:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06f41 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.669062:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e06f41 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27725824->27729919] (req 27725824->27729919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.669314:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493852160:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60041us (60649us total) trans 4294974088 rc 0/0 00000100:00100000:2.0:1789760786.669327:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6799 00000100:00100000:2.0:1789760786.669339:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796878244517 time 796950288914 state 0x2 ref 0 00000100:00100000:2.0:1789760786.683212:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760786.685709:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493852288 00000100:00100000:0.0:1789760786.685730:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493852288 00000100:00100000:0.0:1789760786.686063:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796878244517 time 796950288914 state 0x0 ref 1 00000100:00100000:0.0:1789760786.686083:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796950549330 time 796950288914 state 0x1 ref 1 00000100:00100000:0.0:1789760786.686197:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796950549330 time 797027231911 state 0x0 ref 1 00000100:00100000:0.0:1789760786.686216:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c89fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493852288:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760786.686284:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27729920, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:0.0:1789760786.686294:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06f48 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.686314:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=797s v=(0 0) 00010000:00010000:0.0:1789760786.686320:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.686327:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06f48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.686344:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06f48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.754137:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974089 is committed 00002000:00010000:1.0:1789760786.754320:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27729920 -> 27734016 00010000:00010000:1.0:1789760786.754340:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06f48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.754356:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06f48 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.754367:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06f48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.754376:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760786.754382:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06f48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.754393:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06f48 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.754429:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06f48 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.754438:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e06f48 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27729920->27734015] (req 27729920->27734015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.754557:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c89fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493852288:12345-192.168.203.60@tcp:4:dd.0 Request processed in 68338us (71352us total) trans 4294974089 rc 0/0 00000100:00100000:1.0:1789760786.754574:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6800 00000100:00100000:1.0:1789760786.754587:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796950549330 time 797027231911 state 0x2 ref 0 00000100:00100000:0.0:1789760786.765309:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760786.765433:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493852416 00000100:00100000:3.0:1789760786.765447:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493852416 00000100:00100000:3.0:1789760786.765710:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 796950549330 time 797027231911 state 0x0 ref 1 00000100:00100000:3.0:1789760786.765730:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797027492327 time 797027231911 state 0x1 ref 1 00000100:00100000:3.0:1789760786.765769:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797027492327 time 797106804117 state 0x0 ref 1 00000100:00100000:3.0:1789760786.765783:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493852416:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760786.765854:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27734016, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:3.0:1789760786.765865:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06f4f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.765886:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=797s v=(0 0) 00010000:00010000:3.0:1789760786.765891:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.765899:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06f4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.765921:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06f4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.802622:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974090 is committed 00002000:00010000:1.0:1789760786.802873:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27734016 -> 27738112 00010000:00010000:1.0:1789760786.802895:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06f4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.802910:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06f4f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.802920:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06f4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.802928:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760786.802934:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06f4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.802944:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06f4f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.802998:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06f4f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.803007:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06f4f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27734016->27738111] (req 27734016->27738111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.803133:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493852416:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37345us (37832us total) trans 4294974090 rc 0/0 00000100:00100000:1.0:1789760786.803742:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6801 00000100:00100000:1.0:1789760786.803758:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797027492327 time 797106804117 state 0x2 ref 0 00000100:00100000:0.0:1789760786.807341:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760786.807467:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493852544 00000100:00100000:2.0:1789760786.807480:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493852544 00000100:00100000:2.0:1789760786.807555:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797027492327 time 797106804117 state 0x0 ref 1 00000100:00100000:2.0:1789760786.807573:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797107064533 time 797106804117 state 0x1 ref 1 00000100:00100000:2.0:1789760786.807645:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797107064533 time 797148679242 state 0x0 ref 1 00000100:00100000:2.0:1789760786.807658:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493852544:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760786.807725:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27738112, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:2.0:1789760786.807736:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f56 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.807755:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=797s v=(0 0) 00010000:00010000:2.0:1789760786.807761:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.807767:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.807785:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760786.848530:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27738112 -> 27742208 00010000:00010000:1.0:1789760786.848555:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.848571:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f56 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.848580:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f56 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.848587:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760786.848592:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f56 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.848602:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f56 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.848631:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f56 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.848643:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e06f56 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27738112->27742207] (req 27738112->27742207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.848771:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493852544:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41108us (41438us total) trans 4294974091 rc 0/0 00000100:00100000:1.0:1789760786.848782:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6802 00000100:00100000:1.0:1789760786.848936:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797107064533 time 797148679242 state 0x2 ref 0 00000001:00080000:0.0:1789760786.850403:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974091 is committed 00000100:00100000:2.0:1789760786.854331:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760786.854526:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493852672 00000100:00100000:1.0:1789760786.854539:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493852672 00000100:00100000:1.0:1789760786.854730:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797107064533 time 797148679242 state 0x0 ref 1 00000100:00100000:1.0:1789760786.854752:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797148939658 time 797148679242 state 0x1 ref 1 00000100:00100000:1.0:1789760786.854790:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797148939658 time 797195824742 state 0x0 ref 1 00000100:00100000:1.0:1789760786.854928:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493852672:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760786.855028:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27742208, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:1.0:1789760786.855039:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f5d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.855058:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=797s v=(0 0) 00010000:00010000:1.0:1789760786.855063:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.855070:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.855092:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.902660:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974092 is committed 00002000:00010000:1.0:1789760786.902854:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27742208 -> 27746304 00010000:00010000:1.0:1789760786.902875:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.902890:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f5d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.902901:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f5d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.902910:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760786.902915:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f5d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.902926:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f5d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.902953:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f5d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.902998:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f5d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27742208->27746303] (req 27742208->27746303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.903151:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493852672:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48213us (48818us total) trans 4294974092 rc 0/0 00000100:00100000:1.0:1789760786.903164:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6803 00000100:00100000:1.0:1789760786.903177:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797148939658 time 797195824742 state 0x2 ref 0 00000100:00100000:3.0:1789760786.905689:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760786.905780:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493852800 00000100:00100000:1.0:1789760786.905791:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493852800 00000100:00100000:1.0:1789760786.905862:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797148939658 time 797195824742 state 0x0 ref 1 00000100:00100000:1.0:1789760786.905881:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797196085158 time 797195824742 state 0x1 ref 1 00000100:00100000:1.0:1789760786.905916:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797196085158 time 797246951273 state 0x0 ref 1 00000100:00100000:1.0:1789760786.905929:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493852800:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760786.906189:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27746304, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:1.0:1789760786.906259:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06f64 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.906279:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=797s v=(0 0) 00010000:00010000:1.0:1789760786.906284:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.906291:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06f64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.906308:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06f64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.935474:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974093 is committed 00002000:00010000:0.0:1789760786.935875:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27746304 -> 27750400 00010000:00010000:0.0:1789760786.935893:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06f64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.935908:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06f64 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.935923:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06f64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.935932:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760786.935938:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06f64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.935949:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06f64 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.935991:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06f64 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.936001:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e06f64 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27746304->27750399] (req 27746304->27750399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.936246:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493852800:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30255us (30504us total) trans 4294974093 rc 0/0 00000100:00100000:0.0:1789760786.936261:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6804 00000100:00100000:0.0:1789760786.936273:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797196085158 time 797246951273 state 0x2 ref 0 00000100:00100000:0.0:1789760786.939266:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760786.939943:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493852928 00000100:00100000:3.0:1789760786.939994:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493852928 00000100:00100000:3.0:1789760786.940070:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797196085158 time 797246951273 state 0x0 ref 1 00000100:00100000:3.0:1789760786.940090:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797247211689 time 797246951273 state 0x1 ref 1 00000100:00100000:3.0:1789760786.940278:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797247211689 time 797281312009 state 0x0 ref 1 00000100:00100000:3.0:1789760786.940294:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493852928:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760786.940357:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27750400, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:3.0:1789760786.940369:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06f6b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.940386:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=797s v=(0 0) 00010000:00010000:3.0:1789760786.940391:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.940398:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06f6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.940415:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06f6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.978379:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974094 is committed 00002000:00010000:1.0:1789760786.978724:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27750400 -> 27754496 00010000:00010000:1.0:1789760786.978744:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06f6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.978759:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06f6b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.978769:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06f6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.978777:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760786.978783:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06f6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.978794:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06f6b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.978821:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06f6b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.978831:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06f6b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27750400->27754495] (req 27750400->27754495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.979116:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493852928:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38817us (39855us total) trans 4294974094 rc 0/0 00000100:00100000:1.0:1789760786.979133:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6805 00000100:00100000:1.0:1789760786.979155:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797247211689 time 797281312009 state 0x2 ref 0 00000100:00100000:2.0:1789760786.983422:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760786.983523:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493853056 00000100:00100000:1.0:1789760786.983536:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493853056 00000100:00100000:1.0:1789760786.983610:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797247211689 time 797281312009 state 0x0 ref 1 00000100:00100000:1.0:1789760786.983640:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797281572425 time 797281312009 state 0x1 ref 1 00000100:00100000:1.0:1789760786.983672:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797281572425 time 797324707632 state 0x0 ref 1 00000100:00100000:1.0:1789760786.983687:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493853056:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760786.983751:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27754496, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:1.0:1789760786.983762:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f72 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.983780:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=797s v=(0 0) 00010000:00010000:1.0:1789760786.983786:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.983793:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.983809:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.004443:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760787.035994:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974095 is committed 00002000:00010000:1.0:1789760787.037677:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27754496 -> 27758592 00010000:00010000:1.0:1789760787.037695:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.037709:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f72 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.037719:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.037727:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760787.037731:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.037740:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f72 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.037759:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f72 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.037768:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f72 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27754496->27758591] (req 27754496->27758591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.037883:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493853056:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54194us (54472us total) trans 4294974095 rc 0/0 00000100:00100000:1.0:1789760787.037895:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6806 00000100:00100000:1.0:1789760787.037905:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797281572425 time 797324707632 state 0x2 ref 0 00000100:00100000:3.0:1789760787.044336:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760787.044416:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493853184 00000100:00100000:2.0:1789760787.044426:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493853184 00000100:00100000:2.0:1789760787.044501:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797281572425 time 797324707632 state 0x0 ref 1 00000100:00100000:2.0:1789760787.044517:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797324968048 time 797324707632 state 0x1 ref 1 00000100:00100000:2.0:1789760787.044545:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797324968048 time 797385580589 state 0x0 ref 1 00000100:00100000:2.0:1789760787.044560:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493853184:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760787.044609:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27758592, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:2.0:1789760787.044619:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06f79 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.044634:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=797s v=(0 0) 00010000:00010000:2.0:1789760787.044639:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.044645:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06f79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.044660:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06f79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760787.044945:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760787.070536:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974096 is committed 00002000:00010000:1.0:1789760787.070745:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27758592 -> 27762688 00010000:00010000:1.0:1789760787.070759:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06f79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.070771:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06f79 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.070779:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06f79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.070786:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760787.070789:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06f79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.070797:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06f79 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.070815:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06f79 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.070821:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e06f79 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27758592->27762687] (req 27758592->27762687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.071254:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493853184:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26460us (26831us total) trans 4294974096 rc 0/0 00000100:00100000:1.0:1789760787.071517:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6807 00000100:00100000:1.0:1789760787.071528:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797324968048 time 797385580589 state 0x2 ref 0 00000100:00100000:0.0:1789760787.076855:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760787.082298:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493853312 00000100:00100000:1.0:1789760787.082313:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493853312 00000100:00100000:1.0:1789760787.082442:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797324968048 time 797385580589 state 0x0 ref 1 00000100:00100000:1.0:1789760787.082460:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797385841005 time 797385580589 state 0x1 ref 1 00000100:00100000:1.0:1789760787.082499:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797385841005 time 797423534201 state 0x0 ref 1 00000100:00100000:1.0:1789760787.082513:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4badc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493853312:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760787.082563:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27762688, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:1.0:1789760787.082572:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f80 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.082586:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=797s v=(0 0) 00010000:00010000:1.0:1789760787.082590:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.082597:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.082611:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.120902:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974097 is committed 00002000:00010000:1.0:1789760787.121009:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27762688 -> 27766784 00010000:00010000:1.0:1789760787.121024:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.121038:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f80 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.121047:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.121054:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760787.121059:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.121068:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f80 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.121087:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f80 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.121094:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e06f80 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27762688->27766783] (req 27762688->27766783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.121238:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4badc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493853312:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38723us (44387us total) trans 4294974097 rc 0/0 00000100:00100000:1.0:1789760787.121250:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6808 00000100:00100000:1.0:1789760787.121260:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797385841005 time 797423534201 state 0x2 ref 0 00000100:00100000:2.0:1789760787.126036:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760787.126309:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493853440 00000100:00100000:3.0:1789760787.126320:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493853440 00000100:00100000:3.0:1789760787.126375:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797385841005 time 797423534201 state 0x0 ref 1 00000100:00100000:3.0:1789760787.126390:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797423794617 time 797423534201 state 0x1 ref 1 00000100:00100000:3.0:1789760787.126481:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797423794617 time 797467515663 state 0x0 ref 1 00000100:00100000:3.0:1789760787.126494:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb15050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493853440:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760787.126543:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27766784, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:3.0:1789760787.126552:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06f87 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.126567:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=797s v=(0 0) 00010000:00010000:3.0:1789760787.126572:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.126579:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06f87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.126593:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06f87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.158930:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974098 is committed 00002000:00010000:2.0:1789760787.159030:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27766784 -> 27770880 00010000:00010000:2.0:1789760787.159044:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06f87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.159057:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06f87 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.159065:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06f87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.159072:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760787.159076:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06f87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.159085:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06f87 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.159103:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06f87 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.159110:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e06f87 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27766784->27770879] (req 27766784->27770879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.159316:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb15050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493853440:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32818us (33284us total) trans 4294974098 rc 0/0 00000100:00100000:2.0:1789760787.159327:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6809 00000100:00100000:2.0:1789760787.159337:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797423794617 time 797467515663 state 0x2 ref 0 00000100:00100000:0.0:1789760787.166202:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760787.166283:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493853568 00000100:00100000:2.0:1789760787.166293:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493853568 00000100:00100000:2.0:1789760787.166349:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797423794617 time 797467515663 state 0x0 ref 1 00000100:00100000:2.0:1789760787.166365:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797467776079 time 797467515663 state 0x1 ref 1 00000100:00100000:2.0:1789760787.166399:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797467776079 time 797507434689 state 0x0 ref 1 00000100:00100000:2.0:1789760787.166412:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4badc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493853568:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760787.166460:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27770880, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:2.0:1789760787.166470:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06f8e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.166484:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=797s v=(0 0) 00010000:00010000:2.0:1789760787.166489:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.166495:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06f8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.166509:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06f8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760787.208923:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27770880 -> 27774976 00010000:00010000:1.0:1789760787.208944:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06f8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.208989:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06f8e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.209000:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06f8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.209008:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760787.209013:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06f8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.209023:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06f8e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.209043:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06f8e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.209051:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e06f8e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27770880->27774975] (req 27770880->27774975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.209146:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4badc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493853568:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42721us (42950us total) trans 4294974099 rc 0/0 00000100:00100000:1.0:1789760787.209158:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6810 00000100:00100000:1.0:1789760787.209168:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797467776079 time 797507434689 state 0x2 ref 0 00000001:00080000:0.0:1789760787.210004:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974099 is committed 00000100:00100000:2.0:1789760787.213117:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760787.213224:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493853696 00000100:00100000:0.0:1789760787.213234:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493853696 00000100:00100000:0.0:1789760787.213288:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797467776079 time 797507434689 state 0x0 ref 1 00000100:00100000:0.0:1789760787.213303:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797507695105 time 797507434689 state 0x1 ref 1 00000100:00100000:0.0:1789760787.213331:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797507695105 time 797554366930 state 0x0 ref 1 00000100:00100000:0.0:1789760787.213343:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493853696:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760787.213391:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27774976, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:0.0:1789760787.213400:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06f95 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.213415:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=797s v=(0 0) 00010000:00010000:0.0:1789760787.213420:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.213426:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06f95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.213439:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06f95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.242212:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974100 is committed 00002000:00010000:2.0:1789760787.242400:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27774976 -> 27779072 00010000:00010000:2.0:1789760787.242416:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06f95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.242429:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06f95 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.242437:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06f95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.242445:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760787.242450:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06f95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.242459:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06f95 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.242478:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06f95 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.242485:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e06f95 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27774976->27779071] (req 27774976->27779071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.242577:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493853696:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29231us (29465us total) trans 4294974100 rc 0/0 00000100:00100000:2.0:1789760787.242588:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6811 00000100:00100000:2.0:1789760787.242597:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797507695105 time 797554366930 state 0x2 ref 0 00000100:00100000:1.0:1789760787.246461:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760787.246553:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493853824 00000100:00100000:3.0:1789760787.246563:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493853824 00000100:00100000:3.0:1789760787.246631:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797507695105 time 797554366930 state 0x0 ref 1 00000100:00100000:3.0:1789760787.246647:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797554627346 time 797554366930 state 0x1 ref 1 00000100:00100000:3.0:1789760787.246676:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797554627346 time 797587711065 state 0x0 ref 1 00000100:00100000:3.0:1789760787.246689:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc352c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493853824:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760787.246739:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27779072, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:3.0:1789760787.246749:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06f9c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.246764:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=797s v=(0 0) 00010000:00010000:3.0:1789760787.246768:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.246774:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06f9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.246788:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06f9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760787.281110:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27779072 -> 27783168 00010000:00010000:1.0:1789760787.281129:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06f9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.281152:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06f9c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.281161:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06f9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.281169:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760787.281174:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06f9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.281182:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06f9c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.281268:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06f9c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.281277:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e06f9c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27779072->27783167] (req 27779072->27783167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.281408:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc352c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493853824:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34715us (34951us total) trans 4294974101 rc 0/0 00000100:00100000:1.0:1789760787.281420:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6812 00000100:00100000:1.0:1789760787.281430:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797554627346 time 797587711065 state 0x2 ref 0 00000001:00080000:0.0:1789760787.283344:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974101 is committed 00000100:00100000:3.0:1789760787.284354:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760787.284485:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493853952 00000100:00100000:2.0:1789760787.284495:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493853952 00000100:00100000:2.0:1789760787.284549:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797554627346 time 797587711065 state 0x0 ref 1 00000100:00100000:2.0:1789760787.284565:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797587971481 time 797587711065 state 0x1 ref 1 00000100:00100000:2.0:1789760787.284600:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797587971481 time 797625636468 state 0x0 ref 1 00000100:00100000:2.0:1789760787.284612:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbedf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493853952:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760787.284661:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27783168, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:2.0:1789760787.284669:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06fa3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.284685:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=797s v=(0 0) 00010000:00010000:2.0:1789760787.284689:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.284695:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06fa3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.284708:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06fa3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.322254:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974102 is committed 00002000:00010000:3.0:1789760787.322503:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27783168 -> 27787264 00010000:00010000:3.0:1789760787.322637:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06fa3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.322655:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06fa3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.322664:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06fa3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.322671:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760787.322676:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06fa3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.322686:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06fa3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.322704:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06fa3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.322712:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e06fa3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27783168->27787263] (req 27783168->27787263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.322872:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbedf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493853952:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38256us (38518us total) trans 4294974102 rc 0/0 00000100:00100000:3.0:1789760787.322883:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6813 00000100:00100000:3.0:1789760787.322894:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797587971481 time 797625636468 state 0x2 ref 0 00000100:00100000:2.0:1789760787.330026:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760787.330147:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493854080 00000100:00100000:1.0:1789760787.330158:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493854080 00000100:00100000:1.0:1789760787.330213:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797587971481 time 797625636468 state 0x0 ref 1 00000100:00100000:1.0:1789760787.330228:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797625896884 time 797625636468 state 0x1 ref 1 00000100:00100000:1.0:1789760787.330262:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797625896884 time 797671297612 state 0x0 ref 1 00000100:00100000:1.0:1789760787.330274:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493854080:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760787.330320:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27787264, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:1.0:1789760787.330329:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06faa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.330345:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=797s v=(0 0) 00010000:00010000:1.0:1789760787.330350:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.330355:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06faa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.330369:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06faa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.358533:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974103 is committed 00002000:00010000:2.0:1789760787.358672:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27787264 -> 27791360 00010000:00010000:2.0:1789760787.358688:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06faa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.358701:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06faa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.358709:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06faa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.358716:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760787.358721:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06faa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.358730:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06faa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.358750:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06faa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.358757:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06faa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27787264->27791359] (req 27787264->27791359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.358888:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493854080:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28611us (28867us total) trans 4294974103 rc 0/0 00000100:00100000:2.0:1789760787.358899:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6814 00000100:00100000:2.0:1789760787.358908:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797625896884 time 797671297612 state 0x2 ref 0 00000100:00100000:1.0:1789760787.369449:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760787.369562:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493854208 00000100:00100000:0.0:1789760787.369572:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493854208 00000100:00100000:0.0:1789760787.369630:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797625896884 time 797671297612 state 0x0 ref 1 00000100:00100000:0.0:1789760787.369650:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797671558028 time 797671297612 state 0x1 ref 1 00000100:00100000:0.0:1789760787.369785:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797671558028 time 797710818346 state 0x0 ref 1 00000100:00100000:0.0:1789760787.369799:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493854208:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760787.369846:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27791360, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:0.0:1789760787.369855:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06fb1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.369872:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=797s v=(0 0) 00010000:00010000:0.0:1789760787.369876:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.369883:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06fb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.369897:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06fb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.400721:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974104 is committed 00002000:00010000:0.0:1789760787.400846:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27791360 -> 27795456 00010000:00010000:0.0:1789760787.400862:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06fb1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.400876:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06fb1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.400886:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06fb1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.400899:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760787.400904:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06fb1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.400914:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06fb1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.400934:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06fb1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.400942:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e06fb1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27791360->27795455] (req 27791360->27795455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.401062:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493854208:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31261us (31621us total) trans 4294974104 rc 0/0 00000100:00100000:0.0:1789760787.401073:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6815 00000100:00100000:0.0:1789760787.401083:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797671558028 time 797710818346 state 0x2 ref 0 00000100:00100000:3.0:1789760787.408887:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760787.410042:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493854336 00000100:00100000:1.0:1789760787.410053:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493854336 00000100:00100000:1.0:1789760787.410113:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797671558028 time 797710818346 state 0x0 ref 1 00000100:00100000:1.0:1789760787.410128:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797711078762 time 797710818346 state 0x1 ref 1 00000100:00100000:1.0:1789760787.410221:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797711078762 time 797751255427 state 0x0 ref 1 00000100:00100000:1.0:1789760787.410234:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493854336:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760787.410296:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27795456, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:1.0:1789760787.410305:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06fb8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.410322:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=797s v=(0 0) 00010000:00010000:1.0:1789760787.410326:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.410332:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06fb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.410347:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06fb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760787.465767:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27795456 -> 27799552 00010000:00010000:3.0:1789760787.465785:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06fb8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.465799:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06fb8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.465809:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06fb8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.465817:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760787.465822:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06fb8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.465831:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06fb8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.465851:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06fb8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.465858:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e06fb8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27795456->27799551] (req 27795456->27799551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.466362:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493854336:12345-192.168.203.60@tcp:4:dd.0 Request processed in 56125us (57480us total) trans 4294974105 rc 0/0 00000100:00100000:3.0:1789760787.466373:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6816 00000100:00100000:3.0:1789760787.466383:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797711078762 time 797751255427 state 0x2 ref 0 00000001:00080000:3.0:1789760787.466625:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974105 is committed 00000100:00100000:2.0:1789760787.471262:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760787.473081:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493854464 00000100:00100000:3.0:1789760787.473095:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493854464 00000100:00100000:3.0:1789760787.473164:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797711078762 time 797751255427 state 0x0 ref 1 00000100:00100000:3.0:1789760787.473179:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797751515843 time 797751255427 state 0x1 ref 1 00000100:00100000:2.0:1789760787.473519:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797751515843 time 797814552482 state 0x0 ref 1 00000100:00100000:2.0:1789760787.473536:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493854464:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760787.473600:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27799552, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:2.0:1789760787.473609:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06fbf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.473625:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=797s v=(0 0) 00010000:00010000:2.0:1789760787.473630:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.473636:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06fbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.473650:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06fbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760787.514176:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27799552 -> 27803648 00000001:00080000:3.0:1789760787.514256:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974106 is committed 00010000:00010000:2.0:1789760787.514312:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06fbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.514326:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06fbf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.514335:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06fbf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.514349:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760787.514353:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06fbf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.514363:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06fbf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.514381:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06fbf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.514389:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06fbf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27799552->27803647] (req 27799552->27803647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.514519:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493854464:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40982us (43320us total) trans 4294974106 rc 0/0 00000100:00100000:2.0:1789760787.514531:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6817 00000100:00100000:2.0:1789760787.514542:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797751515843 time 797814552482 state 0x2 ref 0 00000100:00100000:1.0:1789760787.517867:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760787.518050:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493854592 00000100:00100000:2.0:1789760787.518060:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493854592 00000100:00100000:2.0:1789760787.518115:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797751515843 time 797814552482 state 0x0 ref 1 00000100:00100000:2.0:1789760787.518129:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797814812898 time 797814552482 state 0x1 ref 1 00000100:00100000:2.0:1789760787.518167:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797814812898 time 797859203356 state 0x0 ref 1 00000100:00100000:2.0:1789760787.518182:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493854592:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760787.518242:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27803648, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:2.0:1789760787.518251:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06fc6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.518268:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=797s v=(0 0) 00010000:00010000:2.0:1789760787.518272:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.518279:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06fc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.518292:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06fc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.549581:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974107 is committed 00002000:00010000:3.0:1789760787.549687:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27803648 -> 27807744 00010000:00010000:3.0:1789760787.549702:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06fc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.549716:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06fc6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.549725:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06fc6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.549733:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760787.549737:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06fc6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.549746:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06fc6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.549764:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06fc6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.549772:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06fc6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27803648->27807743] (req 27803648->27807743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.549864:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493854592:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31679us (32001us total) trans 4294974107 rc 0/0 00000100:00100000:3.0:1789760787.549876:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6818 00000100:00100000:3.0:1789760787.549885:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797814812898 time 797859203356 state 0x2 ref 0 00000100:00100000:3.0:1789760787.554153:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760787.554593:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493854720 00000100:00100000:2.0:1789760787.554604:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493854720 00000100:00100000:2.0:1789760787.554670:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797814812898 time 797859203356 state 0x0 ref 1 00000100:00100000:2.0:1789760787.554687:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797859463772 time 797859203356 state 0x1 ref 1 00000100:00100000:2.0:1789760787.554718:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797859463772 time 797895753343 state 0x0 ref 1 00000100:00100000:2.0:1789760787.554730:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493854720:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760787.554782:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27807744, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:2.0:1789760787.554792:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06fcd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.554816:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=797s v=(0 0) 00010000:00010000:2.0:1789760787.554820:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.554826:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06fcd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.554840:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06fcd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.585854:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974108 is committed 00002000:00010000:2.0:1789760787.585992:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27807744 -> 27811840 00010000:00010000:2.0:1789760787.586009:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06fcd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.586022:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06fcd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.586031:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06fcd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.586039:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760787.586044:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06fcd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.586053:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06fcd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.586072:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06fcd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.586081:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06fcd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27807744->27811839] (req 27807744->27811839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.586201:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493854720:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31457us (32050us total) trans 4294974108 rc 0/0 00000100:00100000:2.0:1789760787.586212:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6819 00000100:00100000:2.0:1789760787.586223:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797859463772 time 797895753343 state 0x2 ref 0 00000100:00100000:2.0:1789760787.590020:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760787.590119:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493854848 00000100:00100000:3.0:1789760787.590128:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493854848 00000100:00100000:3.0:1789760787.590194:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797859463772 time 797895753343 state 0x0 ref 1 00000100:00100000:3.0:1789760787.590209:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797896013759 time 797895753343 state 0x1 ref 1 00000100:00100000:3.0:1789760787.590236:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797896013759 time 797931271731 state 0x0 ref 1 00000100:00100000:3.0:1789760787.590247:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493854848:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760787.590290:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27811840, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xd809 00010000:00010000:3.0:1789760787.590299:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06fd4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.590313:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=797s v=(0 0) 00010000:00010000:3.0:1789760787.590318:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.590324:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06fd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.590337:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06fd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.628809:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974109 is committed 00002000:00010000:2.0:1789760787.628908:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27811840 -> 27815936 00002000:00010000:2.0:1789760787.628913:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760786 -> 1789760787 00002000:00010000:2.0:1789760787.628915:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760786 -> 1789760787 00010000:00010000:2.0:1789760787.628938:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06fd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.628951:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06fd4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.628975:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06fd4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.628983:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760787.628987:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06fd4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.629001:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06fd4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.629020:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06fd4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.629027:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e06fd4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27811840->27815935] (req 27811840->27815935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.629125:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493854848:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38874us (39109us total) trans 4294974109 rc 0/0 00000100:00100000:2.0:1789760787.629145:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6820 00000100:00100000:2.0:1789760787.629154:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797896013759 time 797931271731 state 0x2 ref 0 00000100:00100000:3.0:1789760787.632921:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760787.633057:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493854976 00000100:00100000:2.0:1789760787.633067:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493854976 00000100:00100000:2.0:1789760787.633124:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797896013759 time 797931271731 state 0x0 ref 1 00000100:00100000:2.0:1789760787.633147:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797931532147 time 797931271731 state 0x1 ref 1 00000100:00100000:2.0:1789760787.633171:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797931532147 time 797974206726 state 0x0 ref 1 00000100:00100000:2.0:1789760787.633192:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbeea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493854976:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760787.633244:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27815936, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:2.0:1789760787.633253:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06fdb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.633268:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=797s v=(0 0) 00010000:00010000:2.0:1789760787.633273:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.633279:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06fdb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.633292:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06fdb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760787.670394:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27815936 -> 27820032 00010000:00010000:3.0:1789760787.670413:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06fdb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.670425:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06fdb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.670434:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06fdb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.670441:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760787.670446:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06fdb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.670454:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06fdb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.670476:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06fdb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.670490:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e06fdb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27815936->27820031] (req 27815936->27820031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.671109:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974110 is committed 00000100:00100000:3.0:1789760787.671406:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbeea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493854976:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38218us (38487us total) trans 4294974110 rc 0/0 00000100:00100000:3.0:1789760787.671417:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6821 00000100:00100000:3.0:1789760787.671427:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797931532147 time 797974206726 state 0x2 ref 0 00000100:00100000:1.0:1789760787.674455:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760787.674546:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493855104 00000100:00100000:2.0:1789760787.674556:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493855104 00000100:00100000:2.0:1789760787.674611:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797931532147 time 797974206726 state 0x0 ref 1 00000100:00100000:2.0:1789760787.674637:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797974467142 time 797974206726 state 0x1 ref 1 00000100:00100000:2.0:1789760787.674666:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797974467142 time 798015701273 state 0x0 ref 1 00000100:00100000:2.0:1789760787.674679:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493855104:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760787.674727:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27820032, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:2.0:1789760787.674736:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06fe2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.674751:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=798s v=(0 0) 00010000:00010000:2.0:1789760787.674755:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.674762:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06fe2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.674776:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06fe2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.715067:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974111 is committed 00002000:00010000:1.0:1789760787.715211:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27820032 -> 27824128 00010000:00010000:1.0:1789760787.715227:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06fe2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.715240:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06fe2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.715249:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06fe2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.715257:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760787.715262:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06fe2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.715271:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06fe2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.715291:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06fe2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.715299:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e06fe2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27820032->27824127] (req 27820032->27824127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.715406:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493855104:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40717us (40947us total) trans 4294974111 rc 0/0 00000100:00100000:1.0:1789760787.715420:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6822 00000100:00100000:1.0:1789760787.715430:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797974467142 time 798015701273 state 0x2 ref 0 00000100:00100000:2.0:1789760787.722458:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760787.722762:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493855232 00000100:00100000:3.0:1789760787.722773:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493855232 00000100:00100000:3.0:1789760787.722831:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 797974467142 time 798015701273 state 0x0 ref 1 00000100:00100000:3.0:1789760787.722847:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798015961689 time 798015701273 state 0x1 ref 1 00000100:00100000:3.0:1789760787.722883:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798015961689 time 798063917785 state 0x0 ref 1 00000100:00100000:3.0:1789760787.722896:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493855232:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760787.722957:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27824128, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:3.0:1789760787.723007:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06fe9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.723024:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=798s v=(0 0) 00010000:00010000:3.0:1789760787.723029:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.723038:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06fe9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.723053:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06fe9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.764340:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974112 is committed 00002000:00010000:1.0:1789760787.764389:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27824128 -> 27828224 00010000:00010000:1.0:1789760787.764406:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06fe9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.764426:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06fe9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.764435:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06fe9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.764442:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760787.764446:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06fe9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.764456:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06fe9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.764475:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06fe9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.764482:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e06fe9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27824128->27828223] (req 27824128->27828223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.764577:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493855232:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41679us (42124us total) trans 4294974112 rc 0/0 00000100:00100000:1.0:1789760787.764588:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6823 00000100:00100000:1.0:1789760787.764598:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798015961689 time 798063917785 state 0x2 ref 0 00000100:00100000:0.0:1789760787.772840:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760787.772946:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493855360 00000100:00100000:3.0:1789760787.772957:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493855360 00000100:00100000:2.0:1789760787.773270:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798015961689 time 798063917785 state 0x0 ref 1 00000100:00100000:2.0:1789760787.773285:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798064178201 time 798063917785 state 0x1 ref 1 00000100:00100000:2.0:1789760787.773364:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798064178201 time 798114397831 state 0x0 ref 1 00000100:00100000:2.0:1789760787.773378:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493855360:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760787.773430:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27828224, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:2.0:1789760787.773439:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06ff0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.773456:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=798s v=(0 0) 00010000:00010000:2.0:1789760787.773461:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.773467:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06ff0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.773482:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06ff0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760787.807478:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27828224 -> 27832320 00010000:00010000:2.0:1789760787.807498:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06ff0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.807510:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06ff0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.807519:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06ff0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.807526:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760787.807530:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06ff0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.807539:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06ff0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.807558:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06ff0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.807565:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06ff0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27828224->27832319] (req 27828224->27832319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.807729:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493855360:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34348us (34895us total) trans 4294974113 rc 0/0 00000100:00100000:2.0:1789760787.807739:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6824 00000100:00100000:2.0:1789760787.807749:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798064178201 time 798114397831 state 0x2 ref 0 00000001:00080000:0.0:1789760787.808508:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974113 is committed 00000100:00100000:3.0:1789760787.813458:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760787.813550:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493855488 00000100:00100000:2.0:1789760787.813561:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493855488 00000100:00100000:2.0:1789760787.813618:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798064178201 time 798114397831 state 0x0 ref 1 00000100:00100000:2.0:1789760787.813634:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798114658247 time 798114397831 state 0x1 ref 1 00000100:00100000:2.0:1789760787.813666:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798114658247 time 798154701591 state 0x0 ref 1 00000100:00100000:2.0:1789760787.813679:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493855488:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760787.813733:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27832320, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:2.0:1789760787.813742:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06ff7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.813758:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=798s v=(0 0) 00010000:00010000:2.0:1789760787.813763:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.813769:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06ff7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.813784:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06ff7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760787.854221:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27832320 -> 27836416 00010000:00010000:0.0:1789760787.854242:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06ff7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.854262:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06ff7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.854271:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06ff7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.854277:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760787.854283:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06ff7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.854291:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06ff7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.854310:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06ff7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.854317:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e06ff7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27832320->27836415] (req 27832320->27836415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.854411:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493855488:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40730us (40959us total) trans 4294974114 rc 0/0 00000100:00100000:0.0:1789760787.854422:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6825 00000100:00100000:0.0:1789760787.854432:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798114658247 time 798154701591 state 0x2 ref 0 00000001:00080000:1.0:1789760787.854676:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974114 is committed 00000100:00100000:1.0:1789760787.859382:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760787.859495:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493855616 00000100:00100000:2.0:1789760787.859507:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493855616 00000100:00100000:2.0:1789760787.859568:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798114658247 time 798154701591 state 0x0 ref 1 00000100:00100000:2.0:1789760787.859584:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798154962007 time 798154701591 state 0x1 ref 1 00000100:00100000:2.0:1789760787.859619:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798154962007 time 798200654958 state 0x0 ref 1 00000100:00100000:2.0:1789760787.859636:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493855616:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760787.859688:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27836416, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:2.0:1789760787.859697:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06ffe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.859712:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=798s v=(0 0) 00010000:00010000:2.0:1789760787.859716:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.859722:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06ffe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.859737:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06ffe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760787.912355:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27836416 -> 27840512 00010000:00010000:1.0:1789760787.912374:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06ffe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.912387:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06ffe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.912396:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06ffe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.912404:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760787.912409:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06ffe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.912427:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06ffe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.912446:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06ffe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.912453:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e06ffe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27836416->27840511] (req 27836416->27840511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.912543:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493855616:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52906us (53167us total) trans 4294974115 rc 0/0 00000100:00100000:1.0:1789760787.912553:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6826 00000100:00100000:1.0:1789760787.912562:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798154962007 time 798200654958 state 0x2 ref 0 00000001:00080000:3.0:1789760787.913638:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974115 is committed 00000100:00100000:3.0:1789760787.916351:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760787.916483:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493855744 00000100:00100000:1.0:1789760787.916494:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493855744 00000100:00100000:1.0:1789760787.916546:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798154962007 time 798200654958 state 0x0 ref 1 00000100:00100000:1.0:1789760787.916561:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798200915374 time 798200654958 state 0x1 ref 1 00000100:00100000:1.0:1789760787.916593:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798200915374 time 798257628473 state 0x0 ref 1 00000100:00100000:1.0:1789760787.916606:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493855744:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760787.916664:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27840512, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:1.0:1789760787.916674:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07005 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.916690:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=798s v=(0 0) 00010000:00010000:1.0:1789760787.916694:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.916701:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07005 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.916717:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07005 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.950586:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974116 is committed 00002000:00010000:1.0:1789760787.950695:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27840512 -> 27844608 00010000:00010000:1.0:1789760787.950710:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07005 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.950729:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07005 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.950738:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07005 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.950745:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760787.950749:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07005 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.950758:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07005 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.950777:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07005 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.950784:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07005 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27840512->27844607] (req 27840512->27844607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.950900:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493855744:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34292us (34555us total) trans 4294974116 rc 0/0 00000100:00100000:1.0:1789760787.950909:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6827 00000100:00100000:1.0:1789760787.950919:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798200915374 time 798257628473 state 0x2 ref 0 00000100:00100000:0.0:1789760787.955524:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760787.955696:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493855872 00000100:00100000:3.0:1789760787.955706:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493855872 00000100:00100000:3.0:1789760787.955761:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798200915374 time 798257628473 state 0x0 ref 1 00000100:00100000:3.0:1789760787.955775:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798257888889 time 798257628473 state 0x1 ref 1 00000100:00100000:3.0:1789760787.955848:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798257888889 time 798296882347 state 0x0 ref 1 00000100:00100000:3.0:1789760787.955860:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493855872:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760787.955908:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27844608, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:3.0:1789760787.955917:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0700c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.955932:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=798s v=(0 0) 00010000:00010000:3.0:1789760787.955936:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.955943:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0700c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.955992:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0700c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.990433:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974117 is committed 00002000:00010000:3.0:1789760787.990547:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27844608 -> 27848704 00010000:00010000:3.0:1789760787.990562:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0700c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.990575:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0700c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.990585:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0700c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.990592:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760787.990597:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0700c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.990607:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0700c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.990706:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0700c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.990714:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0700c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27844608->27848703] (req 27844608->27848703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.990823:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493855872:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34959us (35304us total) trans 4294974117 rc 0/0 00000100:00100000:3.0:1789760787.990833:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6828 00000100:00100000:3.0:1789760787.990844:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798257888889 time 798296882347 state 0x2 ref 0 00000100:00100000:2.0:1789760787.993954:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760787.994064:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493856000 00000100:00100000:3.0:1789760787.994076:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493856000 00000100:00100000:3.0:1789760787.994144:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798257888889 time 798296882347 state 0x0 ref 1 00000100:00100000:3.0:1789760787.994160:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798297142763 time 798296882347 state 0x1 ref 1 00000100:00100000:3.0:1789760787.994199:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798297142763 time 798335233908 state 0x0 ref 1 00000100:00100000:3.0:1789760787.994211:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493856000:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760787.994258:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27848704, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:3.0:1789760787.994267:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07013 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.994284:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=798s v=(0 0) 00010000:00010000:3.0:1789760787.994288:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.994294:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07013 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.994309:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07013 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.026322:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760788.031385:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27848704 -> 27852800 00010000:00010000:3.0:1789760788.031406:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07013 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.031420:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07013 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.031429:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07013 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.031437:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760788.031442:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07013 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.031451:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07013 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.031470:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07013 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.031478:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07013 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27848704->27852799] (req 27848704->27852799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.031583:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493856000:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37369us (37637us total) trans 4294974118 rc 0/0 00000100:00100000:3.0:1789760788.031598:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6829 00000100:00100000:3.0:1789760788.031607:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798297142763 time 798335233908 state 0x2 ref 0 00000100:00100000:3.0:1789760788.035720:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760788.035861:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493856128 00000100:00100000:2.0:1789760788.035871:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493856128 00000100:00100000:2.0:1789760788.035925:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798297142763 time 798335233908 state 0x0 ref 1 00000100:00100000:2.0:1789760788.035941:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798335494324 time 798335233908 state 0x1 ref 1 00000100:00100000:2.0:1789760788.035994:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798335494324 time 798377028896 state 0x0 ref 1 00000100:00100000:2.0:1789760788.036007:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493856128:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760788.036061:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27852800, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:2.0:1789760788.036070:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0701a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.036087:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=798s v=(0 0) 00010000:00010000:2.0:1789760788.036092:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.036099:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0701a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.036113:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0701a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.039883:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974118 is committed 00002000:00010000:3.0:1789760788.078344:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27852800 -> 27856896 00010000:00010000:3.0:1789760788.078362:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0701a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.078375:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0701a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.078384:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0701a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.078391:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760788.078397:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0701a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.078406:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0701a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.078423:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0701a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.078430:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246c0d73/0x3eb3dcc0b2e0701a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27852800->27856895] (req 27852800->27856895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.078521:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493856128:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42512us (42805us total) trans 4294974119 rc 0/0 00000100:00100000:3.0:1789760788.078530:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6830 00000100:00100000:3.0:1789760788.078540:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798335494324 time 798377028896 state 0x2 ref 0 00000001:00080000:0.0:1789760788.082335:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974119 is committed 00000100:00100000:2.0:1789760788.082928:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760788.083059:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493856256 00000100:00100000:1.0:1789760788.083068:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493856256 00000100:00100000:1.0:1789760788.083116:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798335494324 time 798377028896 state 0x0 ref 1 00000100:00100000:1.0:1789760788.083370:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798377289312 time 798377028896 state 0x1 ref 1 00000100:00100000:1.0:1789760788.083429:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798377289312 time 798424464345 state 0x0 ref 1 00000100:00100000:1.0:1789760788.083442:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493856256:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760788.083485:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27856896, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:1.0:1789760788.083494:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07021 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.083509:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=798s v=(0 0) 00010000:00010000:1.0:1789760788.083513:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.083519:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07021 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.083532:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07021 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760788.113618:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27856896 -> 27860992 00010000:00010000:2.0:1789760788.113638:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07021 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.113650:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07021 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.113659:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07021 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.113666:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760788.113671:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07021 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.113684:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07021 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.113704:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07021 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.113712:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07021 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27856896->27860991] (req 27856896->27860991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.113808:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493856256:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30363us (30879us total) trans 4294974120 rc 0/0 00000100:00100000:2.0:1789760788.113819:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6831 00000100:00100000:2.0:1789760788.113829:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798377289312 time 798424464345 state 0x2 ref 0 00000001:00080000:3.0:1789760788.116521:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974120 is committed 00000100:00100000:2.0:1789760788.117786:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760788.117868:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493856384 00000100:00100000:3.0:1789760788.117879:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493856384 00000100:00100000:3.0:1789760788.117931:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798377289312 time 798424464345 state 0x0 ref 1 00000100:00100000:3.0:1789760788.117945:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798424724761 time 798424464345 state 0x1 ref 1 00000100:00100000:3.0:1789760788.118000:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798424724761 time 798459035196 state 0x0 ref 1 00000100:00100000:3.0:1789760788.118012:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493856384:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760788.118063:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27860992, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:3.0:1789760788.118072:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07028 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.118088:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=798s v=(0 0) 00010000:00010000:3.0:1789760788.118093:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.118100:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07028 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.118114:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07028 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760788.148901:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27860992 -> 27865088 00010000:00010000:3.0:1789760788.148920:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07028 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.148933:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07028 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.148942:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07028 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.148950:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760788.148955:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07028 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.148983:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07028 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.149004:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07028 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.149011:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07028 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27860992->27865087] (req 27860992->27865087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.149109:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493856384:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31093us (31323us total) trans 4294974121 rc 0/0 00000100:00100000:3.0:1789760788.149123:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6832 00000100:00100000:3.0:1789760788.149143:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798424724761 time 798459035196 state 0x2 ref 0 00000001:00080000:1.0:1789760788.151360:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974121 is committed 00000100:00100000:0.0:1789760788.152399:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760788.152494:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493856512 00000100:00100000:3.0:1789760788.152504:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493856512 00000100:00100000:3.0:1789760788.152558:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798424724761 time 798459035196 state 0x0 ref 1 00000100:00100000:3.0:1789760788.152573:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798459295612 time 798459035196 state 0x1 ref 1 00000100:00100000:3.0:1789760788.152707:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798459295612 time 798493741574 state 0x0 ref 1 00000100:00100000:3.0:1789760788.152720:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493856512:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760788.152771:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27865088, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:3.0:1789760788.152780:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0702f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.152796:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=798s v=(0 0) 00010000:00010000:3.0:1789760788.152801:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.152807:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0702f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.152822:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0702f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760788.186621:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27865088 -> 27869184 00010000:00010000:3.0:1789760788.186639:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0702f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.186653:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0702f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.186662:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0702f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.186669:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760788.186674:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0702f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.186683:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0702f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.186701:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0702f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.186708:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0702f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27865088->27869183] (req 27865088->27869183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.187006:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493856512:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34281us (34606us total) trans 4294974122 rc 0/0 00000100:00100000:3.0:1789760788.187017:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6833 00000100:00100000:3.0:1789760788.187027:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798459295612 time 798493741574 state 0x2 ref 0 00000001:00080000:2.0:1789760788.188354:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974122 is committed 00000100:00100000:0.0:1789760788.192137:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760788.192322:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493856640 00000100:00100000:2.0:1789760788.192331:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493856640 00000100:00100000:2.0:1789760788.192386:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798459295612 time 798493741574 state 0x0 ref 1 00000100:00100000:2.0:1789760788.192399:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798494001990 time 798493741574 state 0x1 ref 1 00000100:00100000:2.0:1789760788.192422:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798494001990 time 798533459526 state 0x0 ref 1 00000100:00100000:2.0:1789760788.192432:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493856640:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760788.192482:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27869184, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:2.0:1789760788.192491:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e07036 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.192507:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=798s v=(0 0) 00010000:00010000:2.0:1789760788.192511:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.192517:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e07036 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.192598:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e07036 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.219250:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974123 is committed 00002000:00010000:3.0:1789760788.219390:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27869184 -> 27873280 00010000:00010000:3.0:1789760788.219406:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e07036 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.219419:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e07036 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.219427:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e07036 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.219435:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760788.219440:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e07036 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.219450:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e07036 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.219467:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e07036 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.219474:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e07036 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27869184->27873279] (req 27869184->27873279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.219557:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493856640:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27121us (27433us total) trans 4294974123 rc 0/0 00000100:00100000:3.0:1789760788.219567:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6834 00000100:00100000:3.0:1789760788.219576:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798494001990 time 798533459526 state 0x2 ref 0 00000100:00100000:1.0:1789760788.223224:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760788.223323:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493856768 00000100:00100000:2.0:1789760788.223333:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493856768 00000100:00100000:2.0:1789760788.223386:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798494001990 time 798533459526 state 0x0 ref 1 00000100:00100000:2.0:1789760788.223399:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798533719942 time 798533459526 state 0x1 ref 1 00000100:00100000:2.0:1789760788.223427:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798533719942 time 798564462762 state 0x0 ref 1 00000100:00100000:2.0:1789760788.223438:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493856768:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760788.223487:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27873280, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:2.0:1789760788.223496:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0703d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.223511:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=798s v=(0 0) 00010000:00010000:2.0:1789760788.223516:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.223522:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0703d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.223536:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0703d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760788.256426:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27873280 -> 27877376 00010000:00010000:2.0:1789760788.256444:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0703d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.256458:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0703d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.256467:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0703d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.256474:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760788.256478:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0703d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.256488:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0703d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.256505:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0703d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.256518:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0703d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27873280->27877375] (req 27873280->27877375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.256609:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493856768:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33167us (33389us total) trans 4294974124 rc 0/0 00000100:00100000:2.0:1789760788.256618:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6835 00000100:00100000:2.0:1789760788.256626:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798533719942 time 798564462762 state 0x2 ref 0 00000001:00080000:0.0:1789760788.258268:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974124 is committed 00000100:00100000:0.0:1789760788.264575:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760788.264758:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493856896 00000100:00100000:3.0:1789760788.264768:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493856896 00000100:00100000:3.0:1789760788.264820:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798533719942 time 798564462762 state 0x0 ref 1 00000100:00100000:3.0:1789760788.264835:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798564723178 time 798564462762 state 0x1 ref 1 00000100:00100000:3.0:1789760788.264897:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798564723178 time 798605930858 state 0x0 ref 1 00000100:00100000:3.0:1789760788.264909:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493856896:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760788.264976:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27877376, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:3.0:1789760788.264985:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07044 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.265001:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=798s v=(0 0) 00010000:00010000:3.0:1789760788.265006:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.265012:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07044 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.265025:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07044 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.300887:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974125 is committed 00002000:00010000:3.0:1789760788.301054:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27877376 -> 27881472 00010000:00010000:3.0:1789760788.301069:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07044 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.301081:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07044 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.301091:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07044 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.301099:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760788.301104:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07044 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.301112:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07044 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.301141:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07044 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.301149:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07044 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27877376->27881471] (req 27877376->27881471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.301307:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493856896:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36392us (36734us total) trans 4294974125 rc 0/0 00000100:00100000:3.0:1789760788.301318:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6836 00000100:00100000:3.0:1789760788.301327:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798564723178 time 798605930858 state 0x2 ref 0 00000100:00100000:3.0:1789760788.304913:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760788.304991:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493857024 00000100:00100000:0.0:1789760788.305002:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493857024 00000100:00100000:0.0:1789760788.305052:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798564723178 time 798605930858 state 0x0 ref 1 00000100:00100000:0.0:1789760788.305067:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798606191274 time 798605930858 state 0x1 ref 1 00000100:00100000:0.0:1789760788.305107:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798606191274 time 798646142860 state 0x0 ref 1 00000100:00100000:0.0:1789760788.305119:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493857024:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760788.305178:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27881472, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:0.0:1789760788.305186:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0704b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.305201:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=798s v=(0 0) 00010000:00010000:0.0:1789760788.305205:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.305212:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0704b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.305225:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0704b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.374930:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974126 is committed 00002000:00010000:3.0:1789760788.375280:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27881472 -> 27885568 00010000:00010000:3.0:1789760788.375296:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0704b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.375317:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0704b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.375327:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0704b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.375334:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760788.375338:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0704b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.375347:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0704b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.375366:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0704b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.375373:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0704b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27881472->27885567] (req 27881472->27885567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.375457:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493857024:12345-192.168.203.60@tcp:4:dd.0 Request processed in 70336us (70550us total) trans 4294974126 rc 0/0 00000100:00100000:3.0:1789760788.375467:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6837 00000100:00100000:3.0:1789760788.375477:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798606191274 time 798646142860 state 0x2 ref 0 00000100:00100000:2.0:1789760788.379249:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760788.379323:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493857152 00000100:00100000:3.0:1789760788.379332:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493857152 00000100:00100000:3.0:1789760788.379383:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798606191274 time 798646142860 state 0x0 ref 1 00000100:00100000:3.0:1789760788.379397:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798646403276 time 798646142860 state 0x1 ref 1 00000100:00100000:3.0:1789760788.379428:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798646403276 time 798720463631 state 0x0 ref 1 00000100:00100000:3.0:1789760788.379441:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493857152:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760788.379489:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27885568, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:3.0:1789760788.379498:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07052 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.379523:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=798s v=(0 0) 00010000:00010000:3.0:1789760788.379528:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.379534:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07052 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.379549:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07052 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.448185:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974127 is committed 00002000:00010000:2.0:1789760788.448296:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27885568 -> 27889664 00010000:00010000:2.0:1789760788.448310:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07052 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.448323:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07052 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.448333:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07052 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.448340:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760788.448344:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07052 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.448353:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07052 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.448374:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07052 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.448382:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07052 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27885568->27889663] (req 27885568->27889663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.448470:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493857152:12345-192.168.203.60@tcp:4:dd.0 Request processed in 69027us (69225us total) trans 4294974127 rc 0/0 00000100:00100000:2.0:1789760788.448482:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6838 00000100:00100000:2.0:1789760788.448502:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798646403276 time 798720463631 state 0x2 ref 0 00000100:00100000:0.0:1789760788.458806:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760788.458985:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493857280 00000100:00100000:2.0:1789760788.458994:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493857280 00000100:00100000:2.0:1789760788.459048:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798646403276 time 798720463631 state 0x0 ref 1 00000100:00100000:2.0:1789760788.459063:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798720724047 time 798720463631 state 0x1 ref 1 00000100:00100000:2.0:1789760788.459093:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798720724047 time 798800129284 state 0x0 ref 1 00000100:00100000:2.0:1789760788.459105:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493857280:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760788.459165:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27889664, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:2.0:1789760788.459174:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e07059 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.459199:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=798s v=(0 0) 00010000:00010000:2.0:1789760788.459205:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.459212:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e07059 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.459226:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e07059 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760788.529657:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27889664 -> 27893760 00010000:00010000:3.0:1789760788.529683:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e07059 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.529697:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e07059 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.529713:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e07059 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.529721:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760788.529727:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e07059 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.529735:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e07059 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.529756:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e07059 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.529764:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e07059 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27889664->27893759] (req 27889664->27893759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.529869:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493857280:12345-192.168.203.60@tcp:4:dd.0 Request processed in 70759us (71067us total) trans 4294974128 rc 0/0 00000100:00100000:3.0:1789760788.529880:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6839 00000100:00100000:3.0:1789760788.529890:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798720724047 time 798800129284 state 0x2 ref 0 00000001:00080000:3.0:1789760788.529932:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974128 is committed 00000100:00100000:2.0:1789760788.533715:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760788.533846:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493857408 00000100:00100000:3.0:1789760788.533856:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493857408 00000100:00100000:3.0:1789760788.533920:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798720724047 time 798800129284 state 0x0 ref 1 00000100:00100000:3.0:1789760788.533940:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798800389700 time 798800129284 state 0x1 ref 1 00000100:00100000:3.0:1789760788.534008:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798800389700 time 798875043396 state 0x0 ref 1 00000100:00100000:3.0:1789760788.534021:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493857408:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760788.534073:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27893760, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:3.0:1789760788.534083:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07060 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.534098:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=798s v=(0 0) 00010000:00010000:3.0:1789760788.534103:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.534109:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07060 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.534123:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07060 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.585854:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760788.585894:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760788.586101:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760788.586138:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493857664 00000100:00100000:0.0:1789760788.586147:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493857664 00000100:00100000:1.0:1789760788.586257:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493857536 00000100:00100000:1.0:1789760788.586261:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493857536 00000100:00100000:1.0:1789760788.586292:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 417 00000100:00100000:1.0:1789760788.586300:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d05c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2370:x1876699493857536:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:1.0:1789760788.586365:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d05c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2370:x1876699493857536:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 65us (520us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760788.586372:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 417 00000100:00080000:1.0:1789760788.586399:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760788.586589:0:9460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493857792 00000100:00100000:0.0:1789760788.586593:0:9460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493857792 00000100:00100000:0.0:1789760788.586621:0:9460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 527 00000100:00100000:0.0:1789760788.586628:0:9460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2369:x1876699493857664:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:0.0:1789760788.586658:0:9460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2369:x1876699493857664:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 30us (763us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760788.586664:0:9460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 527 00000100:00100000:0.0:1789760788.586675:0:9460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 528 00000100:00100000:0.0:1789760788.586680:0:9460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2369:x1876699493857792:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:0.0:1789760788.586696:0:9460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5836300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2369:x1876699493857792:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 17us (597us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760788.586702:0:9460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 528 00000100:00080000:1.0:1789760788.586986:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00002000:00010000:3.0:1789760788.594589:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27893760 -> 27897856 00010000:00010000:3.0:1789760788.594608:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07060 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.594621:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07060 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.594631:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07060 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.594644:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760788.594650:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07060 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.594659:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07060 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.594678:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07060 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.594685:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07060 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27893760->27897855] (req 27893760->27897855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.594774:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493857408:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60751us (61064us total) trans 4294974129 rc 0/0 00000100:00100000:3.0:1789760788.594784:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6840 00000100:00100000:3.0:1789760788.594794:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798800389700 time 798875043396 state 0x2 ref 0 00000001:00080000:2.0:1789760788.595304:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974129 is committed 00000100:00100000:0.0:1789760788.598724:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760788.600840:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493857920 00000100:00100000:3.0:1789760788.600853:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493857920 00000100:00100000:3.0:1789760788.600914:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798800389700 time 798875043396 state 0x0 ref 1 00000100:00100000:3.0:1789760788.600930:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798875303812 time 798875043396 state 0x1 ref 1 00000100:00100000:3.0:1789760788.601053:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798875303812 time 798942088293 state 0x0 ref 1 00000100:00100000:3.0:1789760788.601069:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493857920:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760788.601116:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27897856, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xd809 00010000:00010000:3.0:1789760788.601136:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e07067 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.601151:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=798s v=(0 0) 00010000:00010000:3.0:1789760788.601156:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.601162:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e07067 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.601176:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e07067 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.651043:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974130 is committed 00002000:00010000:2.0:1789760788.651549:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27897856 -> 27901952 00002000:00010000:2.0:1789760788.651555:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760787 -> 1789760788 00002000:00010000:2.0:1789760788.651558:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760787 -> 1789760788 00010000:00010000:2.0:1789760788.651569:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e07067 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.651583:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e07067 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.651593:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e07067 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.651600:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760788.651613:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e07067 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.651622:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e07067 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.651641:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e07067 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.651649:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e07067 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27897856->27901951] (req 27897856->27901951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.651747:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493857920:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50675us (53026us total) trans 4294974130 rc 0/0 00000100:00100000:2.0:1789760788.651759:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6841 00000100:00100000:2.0:1789760788.651770:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798875303812 time 798942088293 state 0x2 ref 0 00000100:00100000:1.0:1789760788.670446:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760788.670541:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493858048 00000100:00100000:2.0:1789760788.670550:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493858048 00000100:00100000:2.0:1789760788.670604:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798875303812 time 798942088293 state 0x0 ref 1 00000100:00100000:2.0:1789760788.670628:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798942348709 time 798942088293 state 0x1 ref 1 00000100:00100000:2.0:1789760788.670655:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798942348709 time 799011691191 state 0x0 ref 1 00000100:00100000:2.0:1789760788.670668:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493858048:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760788.670719:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27901952, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:2.0:1789760788.670728:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0706e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.670744:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=799s v=(0 0) 00010000:00010000:2.0:1789760788.670749:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.670755:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0706e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.670770:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0706e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.714214:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974131 is committed 00002000:00010000:3.0:1789760788.715221:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27901952 -> 27906048 00010000:00010000:3.0:1789760788.715238:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0706e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.715250:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0706e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.715259:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0706e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.715267:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760788.715272:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0706e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.715281:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0706e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.715299:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0706e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.715306:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0706e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27901952->27906047] (req 27901952->27906047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.715547:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493858048:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44876us (45105us total) trans 4294974131 rc 0/0 00000100:00100000:3.0:1789760788.715558:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6842 00000100:00100000:3.0:1789760788.715567:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798942348709 time 799011691191 state 0x2 ref 0 00000100:00100000:1.0:1789760788.722471:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760788.722551:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493858176 00000100:00100000:2.0:1789760788.722559:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493858176 00000100:00100000:2.0:1789760788.722614:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 798942348709 time 799011691191 state 0x0 ref 1 00000100:00100000:2.0:1789760788.722630:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799011951607 time 799011691191 state 0x1 ref 1 00000100:00100000:2.0:1789760788.722662:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799011951607 time 799063698127 state 0x0 ref 1 00000100:00100000:2.0:1789760788.722675:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493858176:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760788.722720:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27906048, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:2.0:1789760788.722729:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e07075 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.722744:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=799s v=(0 0) 00010000:00010000:2.0:1789760788.722749:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.722755:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e07075 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.722769:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e07075 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.766840:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974132 is committed 00002000:00010000:2.0:1789760788.768785:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27906048 -> 27910144 00010000:00010000:2.0:1789760788.768805:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e07075 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.768819:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e07075 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.768829:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e07075 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.768837:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760788.768841:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e07075 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.768851:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e07075 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.768870:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e07075 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.768879:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e07075 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27906048->27910143] (req 27906048->27910143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.769018:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493858176:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46341us (46553us total) trans 4294974132 rc 0/0 00000100:00100000:2.0:1789760788.769032:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6843 00000100:00100000:2.0:1789760788.769042:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799011951607 time 799063698127 state 0x2 ref 0 00000100:00100000:0.0:1789760788.782602:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760788.782696:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493858304 00000100:00100000:3.0:1789760788.782706:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493858304 00000100:00100000:3.0:1789760788.782763:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799011951607 time 799063698127 state 0x0 ref 1 00000100:00100000:3.0:1789760788.782780:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799063958543 time 799063698127 state 0x1 ref 1 00000100:00100000:3.0:1789760788.782816:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799063958543 time 799123850614 state 0x0 ref 1 00000100:00100000:3.0:1789760788.782829:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c87a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493858304:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760788.782892:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27910144, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:3.0:1789760788.782901:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0707c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.782918:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=799s v=(0 0) 00010000:00010000:3.0:1789760788.782923:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.782930:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0707c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.782945:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0707c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.840180:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974133 is committed 00002000:00010000:2.0:1789760788.840329:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27910144 -> 27914240 00010000:00010000:2.0:1789760788.840405:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0707c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.840421:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0707c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.840431:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0707c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.840439:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760788.840444:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0707c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.840453:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0707c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.840472:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0707c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.840480:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e0707c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27910144->27914239] (req 27910144->27914239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.840632:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c87a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493858304:12345-192.168.203.60@tcp:4:dd.0 Request processed in 57800us (58036us total) trans 4294974133 rc 0/0 00000100:00100000:2.0:1789760788.840643:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6844 00000100:00100000:2.0:1789760788.840653:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799063958543 time 799123850614 state 0x2 ref 0 00000100:00100000:1.0:1789760788.844577:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760788.844677:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493858432 00000100:00100000:0.0:1789760788.844686:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493858432 00000100:00100000:0.0:1789760788.844740:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799063958543 time 799123850614 state 0x0 ref 1 00000100:00100000:0.0:1789760788.844753:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799124111030 time 799123850614 state 0x1 ref 1 00000100:00100000:0.0:1789760788.844819:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799124111030 time 799185854032 state 0x0 ref 1 00000100:00100000:0.0:1789760788.844831:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493858432:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760788.844882:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27914240, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:0.0:1789760788.844891:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e07083 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.844906:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=799s v=(0 0) 00010000:00010000:0.0:1789760788.844911:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.844917:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e07083 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.844931:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e07083 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.888407:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974134 is committed 00002000:00010000:2.0:1789760788.888518:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27914240 -> 27918336 00010000:00010000:2.0:1789760788.888596:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e07083 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.888610:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e07083 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.888618:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e07083 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.888627:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760788.888631:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e07083 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.888641:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e07083 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.888660:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e07083 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.888675:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e07083 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27914240->27918335] (req 27914240->27918335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.888799:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493858432:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43964us (44225us total) trans 4294974134 rc 0/0 00000100:00100000:2.0:1789760788.888809:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6845 00000100:00100000:2.0:1789760788.888818:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799124111030 time 799185854032 state 0x2 ref 0 00000100:00100000:1.0:1789760788.897230:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760788.897304:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493858560 00000100:00100000:2.0:1789760788.897314:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493858560 00000100:00100000:2.0:1789760788.897366:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799124111030 time 799185854032 state 0x0 ref 1 00000100:00100000:2.0:1789760788.897380:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799186114448 time 799185854032 state 0x1 ref 1 00000100:00100000:2.0:1789760788.897407:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799186114448 time 799238443165 state 0x0 ref 1 00000100:00100000:2.0:1789760788.897418:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493858560:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760788.897464:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27918336, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:2.0:1789760788.897472:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0708a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.897487:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=799s v=(0 0) 00010000:00010000:2.0:1789760788.897492:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.897497:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0708a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.897515:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0708a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760788.936275:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27918336 -> 27922432 00010000:00010000:3.0:1789760788.936294:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0708a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.936308:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0708a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.936316:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0708a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.936323:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760788.936328:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0708a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.936337:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0708a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.936365:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0708a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.936373:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0708a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27918336->27922431] (req 27918336->27922431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.936464:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493858560:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39043us (39239us total) trans 4294974135 rc 0/0 00000100:00100000:3.0:1789760788.936472:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6846 00000100:00100000:3.0:1789760788.936480:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799186114448 time 799238443165 state 0x2 ref 0 00000001:00080000:1.0:1789760788.936556:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974135 is committed 00000100:00100000:2.0:1789760788.940462:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760788.942422:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493858688 00000100:00100000:0.0:1789760788.942435:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493858688 00000100:00100000:0.0:1789760788.942496:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799186114448 time 799238443165 state 0x0 ref 1 00000100:00100000:0.0:1789760788.942511:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799238703581 time 799238443165 state 0x1 ref 1 00000100:00100000:0.0:1789760788.942539:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799238703581 time 799283575291 state 0x0 ref 1 00000100:00100000:0.0:1789760788.942552:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493858688:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760788.942608:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27922432, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:0.0:1789760788.942617:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07091 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.942634:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=799s v=(0 0) 00010000:00010000:0.0:1789760788.942638:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.942644:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07091 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.942659:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07091 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.975798:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974136 is committed 00002000:00010000:2.0:1789760788.975956:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27922432 -> 27926528 00010000:00010000:2.0:1789760788.975994:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07091 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.976005:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07091 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.976014:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07091 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.976022:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760788.976027:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07091 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.976036:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07091 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.976055:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07091 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.976063:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07091 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27922432->27926527] (req 27922432->27926527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.976182:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493858688:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33627us (35723us total) trans 4294974136 rc 0/0 00000100:00100000:2.0:1789760788.976191:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6847 00000100:00100000:2.0:1789760788.976200:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799238703581 time 799283575291 state 0x2 ref 0 00000100:00100000:2.0:1789760788.982101:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760788.983914:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493858816 00000100:00100000:3.0:1789760788.983929:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493858816 00000100:00100000:3.0:1789760788.984036:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799238703581 time 799283575291 state 0x0 ref 1 00000100:00100000:3.0:1789760788.984053:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799283835707 time 799283575291 state 0x1 ref 1 00000100:00100000:3.0:1789760788.984092:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799283835707 time 799325127525 state 0x0 ref 1 00000100:00100000:3.0:1789760788.984106:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493858816:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760788.984179:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27926528, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:3.0:1789760788.984188:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e07098 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.984208:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=799s v=(0 0) 00010000:00010000:3.0:1789760788.984213:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.984220:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e07098 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.984234:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e07098 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.042998:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974137 is committed 00002000:00010000:2.0:1789760789.043131:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27926528 -> 27930624 00010000:00010000:2.0:1789760789.043147:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e07098 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.043159:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e07098 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.043168:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e07098 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.043182:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760789.043187:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e07098 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.043196:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e07098 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.043215:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e07098 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.043228:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e07098 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27926528->27930623] (req 27926528->27930623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.043318:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493858816:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59211us (61228us total) trans 4294974137 rc 0/0 00000100:00100000:2.0:1789760789.043330:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6848 00000100:00100000:2.0:1789760789.043339:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799283835707 time 799325127525 state 0x2 ref 0 00000100:00100000:1.0:1789760789.048314:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760789.048406:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493858944 00000100:00100000:2.0:1789760789.048415:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493858944 00000100:00100000:2.0:1789760789.048621:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799283835707 time 799325127525 state 0x0 ref 1 00000100:00100000:2.0:1789760789.048637:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799325387941 time 799325127525 state 0x1 ref 1 00000100:00100000:2.0:1789760789.048668:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799325387941 time 799389703712 state 0x0 ref 1 00000100:00100000:2.0:1789760789.048682:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a37480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493858944:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760789.048730:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27930624, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:2.0:1789760789.048739:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0709f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.048754:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=799s v=(0 0) 00010000:00010000:2.0:1789760789.048759:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.048765:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0709f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.048783:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0709f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760789.049396:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1789760789.051161:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760789.092788:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974138 is committed 00002000:00010000:3.0:1789760789.092900:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27930624 -> 27934720 00010000:00010000:3.0:1789760789.092917:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0709f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.092930:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0709f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.092940:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0709f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.092948:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760789.092952:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0709f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.092982:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0709f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.093006:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0709f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.093015:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e0709f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27930624->27934719] (req 27930624->27934719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.093132:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a37480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493858944:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44440us (44815us total) trans 4294974138 rc 0/0 00000100:00100000:3.0:1789760789.093142:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6849 00000100:00100000:3.0:1789760789.093150:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799325387941 time 799389703712 state 0x2 ref 0 00000100:00100000:3.0:1789760789.103769:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760789.104057:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493859072 00000100:00100000:2.0:1789760789.104067:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493859072 00000100:00100000:2.0:1789760789.104141:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799325387941 time 799389703712 state 0x0 ref 1 00000100:00100000:2.0:1789760789.104155:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799389964128 time 799389703712 state 0x1 ref 1 00000100:00100000:2.0:1789760789.104188:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799389964128 time 799445224046 state 0x0 ref 1 00000100:00100000:2.0:1789760789.104200:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3663480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493859072:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760789.104244:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27934720, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:2.0:1789760789.104253:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e070a6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.104266:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=799s v=(0 0) 00010000:00010000:2.0:1789760789.104271:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.104276:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e070a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.104290:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e070a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.153718:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974139 is committed 00002000:00010000:1.0:1789760789.153851:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27934720 -> 27938816 00010000:00010000:1.0:1789760789.153867:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e070a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.153879:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e070a6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.153887:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e070a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.153893:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760789.153897:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e070a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.153905:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e070a6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.153922:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e070a6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.153929:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e070a6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27934720->27938815] (req 27934720->27938815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.154151:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3663480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493859072:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49948us (50385us total) trans 4294974139 rc 0/0 00000100:00100000:1.0:1789760789.154162:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6850 00000100:00100000:1.0:1789760789.154172:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799389964128 time 799445224046 state 0x2 ref 0 00000100:00100000:2.0:1789760789.157554:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760789.157655:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493859200 00000100:00100000:0.0:1789760789.157665:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493859200 00000100:00100000:0.0:1789760789.157874:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799389964128 time 799445224046 state 0x0 ref 1 00000100:00100000:0.0:1789760789.157891:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799445484462 time 799445224046 state 0x1 ref 1 00000100:00100000:0.0:1789760789.157921:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799445484462 time 799498957228 state 0x0 ref 1 00000100:00100000:0.0:1789760789.157935:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493859200:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760789.158014:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27938816, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:0.0:1789760789.158023:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e070ad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.158038:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=799s v=(0 0) 00010000:00010000:0.0:1789760789.158043:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.158049:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e070ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.158063:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e070ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.209615:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974140 is committed 00002000:00010000:3.0:1789760789.212230:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27938816 -> 27942912 00010000:00010000:3.0:1789760789.212249:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e070ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.212262:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e070ad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.212271:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e070ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.212279:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760789.212284:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e070ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.212298:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e070ad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.212318:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e070ad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.212326:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e070ad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27938816->27942911] (req 27938816->27942911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.212423:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493859200:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54486us (54874us total) trans 4294974140 rc 0/0 00000100:00100000:3.0:1789760789.212433:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6851 00000100:00100000:3.0:1789760789.212450:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799445484462 time 799498957228 state 0x2 ref 0 00000100:00100000:2.0:1789760789.219238:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760789.219335:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493859328 00000100:00100000:3.0:1789760789.219353:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493859328 00000100:00100000:3.0:1789760789.219410:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799445484462 time 799498957228 state 0x0 ref 1 00000100:00100000:3.0:1789760789.219426:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799499217644 time 799498957228 state 0x1 ref 1 00000100:00100000:3.0:1789760789.219456:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799499217644 time 799560491999 state 0x0 ref 1 00000100:00100000:3.0:1789760789.219467:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493859328:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760789.219518:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27942912, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:3.0:1789760789.219528:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e070b4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.219543:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=799s v=(0 0) 00010000:00010000:3.0:1789760789.219547:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.219553:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e070b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.219568:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e070b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760789.287358:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27942912 -> 27947008 00010000:00010000:0.0:1789760789.287378:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e070b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.287393:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e070b4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.287401:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e070b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.287409:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760789.287414:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e070b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.287424:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e070b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.287443:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e070b4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.287450:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e070b4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27942912->27947007] (req 27942912->27947007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.287543:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493859328:12345-192.168.203.60@tcp:4:dd.0 Request processed in 68072us (68309us total) trans 4294974141 rc 0/0 00000100:00100000:0.0:1789760789.287714:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6852 00000100:00100000:0.0:1789760789.287724:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799499217644 time 799560491999 state 0x2 ref 0 00000001:00080000:0.0:1789760789.287763:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974141 is committed 00000100:00100000:2.0:1789760789.291611:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760789.291791:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493859456 00000100:00100000:3.0:1789760789.291801:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493859456 00000100:00100000:3.0:1789760789.291859:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799499217644 time 799560491999 state 0x0 ref 1 00000100:00100000:3.0:1789760789.291876:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799560752415 time 799560491999 state 0x1 ref 1 00000100:00100000:3.0:1789760789.291906:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799560752415 time 799632941520 state 0x0 ref 1 00000100:00100000:3.0:1789760789.291919:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493859456:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760789.292018:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27947008, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:3.0:1789760789.292028:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e070bb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.292043:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=799s v=(0 0) 00010000:00010000:3.0:1789760789.292048:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.292054:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e070bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.292073:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e070bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.331819:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974142 is committed 00002000:00010000:2.0:1789760789.331905:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27947008 -> 27951104 00010000:00010000:2.0:1789760789.331921:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e070bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.331934:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e070bb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.331943:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e070bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.331950:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760789.331955:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e070bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.332471:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e070bb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.332491:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e070bb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.332500:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e070bb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27947008->27951103] (req 27947008->27951103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.332601:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493859456:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40679us (41003us total) trans 4294974142 rc 0/0 00000100:00100000:2.0:1789760789.332621:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6853 00000100:00100000:2.0:1789760789.332630:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799560752415 time 799632941520 state 0x2 ref 0 00000100:00100000:3.0:1789760789.336709:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760789.336778:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493859584 00000100:00100000:1.0:1789760789.336789:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493859584 00000100:00100000:1.0:1789760789.336850:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799560752415 time 799632941520 state 0x0 ref 1 00000100:00100000:1.0:1789760789.336866:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799633201936 time 799632941520 state 0x1 ref 1 00000100:00100000:1.0:1789760789.336898:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799633201936 time 799677932872 state 0x0 ref 1 00000100:00100000:1.0:1789760789.336912:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3662d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493859584:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760789.336981:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27951104, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:1.0:1789760789.336991:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070c2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.337007:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=799s v=(0 0) 00010000:00010000:1.0:1789760789.337011:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.337017:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.337031:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.373241:0:3304:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760789.373263:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc48dea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3304:1876699519261568:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760789.373333:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760789.373553:0:6691:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519261568 00000100:00100000:2.0:1789760789.373561:0:6691:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519261568 00000100:00100000:2.0:1789760789.373620:0:6691:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 218 00000100:00100000:2.0:1789760789.373630:0:6691:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519261568:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760789.373651:0:6691:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528154624 pending 1703936 free 3830448128 avail 3815768064 00000100:00100000:2.0:1789760789.373692:0:6691:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3304:x1876699519261568:12345-0@lo:13:osp-pre-0-0.0 Request processed in 64us (360us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760789.373709:0:6691:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 218 00000100:00080000:2.0:1789760789.373730:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760789.373761:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc48dea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3304:1876699519261568:0@lo:13:osp-pre-0-0.0 00002000:00010000:1.0:1789760789.382099:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27951104 -> 27955200 00010000:00010000:1.0:1789760789.382118:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.382142:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070c2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.382151:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.382160:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760789.382164:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.382184:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070c2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.382208:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070c2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.382217:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070c2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27951104->27955199] (req 27951104->27955199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.382298:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3662d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493859584:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45386us (45597us total) trans 4294974143 rc 0/0 00000100:00100000:1.0:1789760789.382309:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6854 00000100:00100000:1.0:1789760789.382319:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799633201936 time 799677932872 state 0x2 ref 0 00000001:00080000:0.0:1789760789.382536:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974143 is committed 00000100:00100000:3.0:1789760789.387473:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760789.388526:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493859712 00000100:00100000:0.0:1789760789.388538:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493859712 00000100:00100000:0.0:1789760789.388596:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799633201936 time 799677932872 state 0x0 ref 1 00000100:00100000:0.0:1789760789.388611:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799678193288 time 799677932872 state 0x1 ref 1 00000100:00100000:0.0:1789760789.388646:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799678193288 time 799729680966 state 0x0 ref 1 00000100:00100000:0.0:1789760789.388658:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3662680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493859712:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760789.388705:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27955200, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:0.0:1789760789.388714:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e070c9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.388729:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=799s v=(0 0) 00010000:00010000:0.0:1789760789.388733:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.388739:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e070c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.388758:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e070c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.432624:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974144 is committed 00002000:00010000:3.0:1789760789.432771:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27955200 -> 27959296 00010000:00010000:3.0:1789760789.432786:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e070c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.432799:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e070c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.432808:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e070c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.432815:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760789.432820:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e070c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.432830:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e070c9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.432851:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e070c9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.432866:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e070c9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27955200->27959295] (req 27955200->27959295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.433411:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3662680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493859712:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44749us (45942us total) trans 4294974144 rc 0/0 00000100:00100000:3.0:1789760789.433422:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6855 00000100:00100000:3.0:1789760789.433431:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799678193288 time 799729680966 state 0x2 ref 0 00000100:00100000:2.0:1789760789.444705:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760789.448591:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493859840 00000100:00100000:0.0:1789760789.448606:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493859840 00000100:00100000:0.0:1789760789.448671:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799678193288 time 799729680966 state 0x0 ref 1 00000100:00100000:0.0:1789760789.448686:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799729941382 time 799729680966 state 0x1 ref 1 00000100:00100000:0.0:1789760789.448711:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799729941382 time 799789746252 state 0x0 ref 1 00000100:00100000:0.0:1789760789.448725:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493859840:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760789.448773:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27959296, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:0.0:1789760789.448782:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e070d0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.448799:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=799s v=(0 0) 00010000:00010000:0.0:1789760789.448804:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.448810:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e070d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.448825:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e070d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.477822:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974145 is committed 00002000:00010000:3.0:1789760789.477925:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27959296 -> 27963392 00010000:00010000:3.0:1789760789.477941:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e070d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.477954:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e070d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.477993:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e070d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.478000:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760789.478004:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e070d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.478013:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e070d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.478037:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e070d0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.478044:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e070d0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27959296->27963391] (req 27959296->27963391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.478144:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493859840:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29418us (33450us total) trans 4294974145 rc 0/0 00000100:00100000:3.0:1789760789.478155:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6856 00000100:00100000:3.0:1789760789.478165:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799729941382 time 799789746252 state 0x2 ref 0 00000100:00100000:2.0:1789760789.482620:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760789.482752:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493859968 00000100:00100000:3.0:1789760789.482761:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493859968 00000100:00100000:3.0:1789760789.482813:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799729941382 time 799789746252 state 0x0 ref 1 00000100:00100000:3.0:1789760789.482828:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799790006668 time 799789746252 state 0x1 ref 1 00000100:00100000:3.0:1789760789.482855:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799790006668 time 799823890973 state 0x0 ref 1 00000100:00100000:3.0:1789760789.482868:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb452a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493859968:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760789.482914:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27963392, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:3.0:1789760789.482922:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e070d7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.482936:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=799s v=(0 0) 00010000:00010000:3.0:1789760789.482942:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.482948:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e070d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.482991:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e070d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.568787:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974146 is committed 00002000:00010000:1.0:1789760789.569193:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27963392 -> 27967488 00010000:00010000:1.0:1789760789.569275:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e070d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.569290:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e070d7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.569298:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e070d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.569306:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760789.569311:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e070d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.569327:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e070d7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.569345:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e070d7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.569352:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e070d7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27963392->27967487] (req 27963392->27967487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.569456:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb452a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493859968:12345-192.168.203.60@tcp:4:dd.0 Request processed in 86584us (86840us total) trans 4294974146 rc 0/0 00000100:00100000:1.0:1789760789.569467:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6857 00000100:00100000:1.0:1789760789.569479:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799790006668 time 799823890973 state 0x2 ref 0 00000100:00100000:2.0:1789760789.575645:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760789.576017:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493860096 00000100:00100000:1.0:1789760789.576027:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493860096 00000100:00100000:1.0:1789760789.576088:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799790006668 time 799823890973 state 0x0 ref 1 00000100:00100000:1.0:1789760789.576102:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799824151389 time 799823890973 state 0x1 ref 1 00000100:00100000:1.0:1789760789.576144:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799824151389 time 799917179612 state 0x0 ref 1 00000100:00100000:1.0:1789760789.576156:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb453800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493860096:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760789.576359:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27967488, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xd809 00010000:00010000:1.0:1789760789.576370:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070de lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.576385:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=799s v=(0 0) 00010000:00010000:1.0:1789760789.576389:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.576396:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.576410:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.647789:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974147 is committed 00002000:00010000:1.0:1789760789.647826:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27967488 -> 27971584 00002000:00010000:1.0:1789760789.647831:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760788 -> 1789760789 00002000:00010000:1.0:1789760789.647833:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760788 -> 1789760789 00010000:00010000:1.0:1789760789.647847:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.647859:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070de lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.647869:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.647875:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760789.647880:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.647890:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.647910:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070de lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.647918:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070de lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27967488->27971583] (req 27967488->27971583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.648046:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb453800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493860096:12345-192.168.203.60@tcp:4:dd.0 Request processed in 71887us (72406us total) trans 4294974147 rc 0/0 00000100:00100000:1.0:1789760789.648056:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6858 00000100:00100000:1.0:1789760789.648065:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799824151389 time 799917179612 state 0x2 ref 0 00000100:00100000:2.0:1789760789.655447:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760789.655625:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493860224 00000100:00100000:0.0:1789760789.655635:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493860224 00000100:00100000:0.0:1789760789.655698:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799824151389 time 799917179612 state 0x0 ref 1 00000100:00100000:0.0:1789760789.655714:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799917440028 time 799917179612 state 0x1 ref 1 00000100:00100000:0.0:1789760789.655747:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799917440028 time 799996782095 state 0x0 ref 1 00000100:00100000:0.0:1789760789.655759:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493860224:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760789.655810:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27971584, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:0.0:1789760789.655819:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e070e5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.655835:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=799s v=(0 0) 00010000:00010000:0.0:1789760789.655839:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.655847:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e070e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.655861:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e070e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.715526:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974148 is committed 00002000:00010000:1.0:1789760789.715680:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27971584 -> 27975680 00010000:00010000:1.0:1789760789.715702:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e070e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.715715:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e070e5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.715724:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e070e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.715732:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760789.715737:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e070e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.715751:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e070e5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.715773:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e070e5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.715778:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e070e5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27971584->27975679] (req 27971584->27975679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.715875:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493860224:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60114us (60434us total) trans 4294974148 rc 0/0 00000100:00100000:1.0:1789760789.715882:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6859 00000100:00100000:1.0:1789760789.715890:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799917440028 time 799996782095 state 0x2 ref 0 00000100:00100000:0.0:1789760789.719608:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760789.719705:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493860352 00000100:00100000:3.0:1789760789.719715:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493860352 00000100:00100000:3.0:1789760789.719768:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799917440028 time 799996782095 state 0x0 ref 1 00000100:00100000:3.0:1789760789.719786:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799997042511 time 799996782095 state 0x1 ref 1 00000100:00100000:3.0:1789760789.719858:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799997042511 time 800060892259 state 0x0 ref 1 00000100:00100000:3.0:1789760789.719873:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493860352:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760789.719922:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27975680, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:3.0:1789760789.719931:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e070ec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.719947:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=800s v=(0 0) 00010000:00010000:3.0:1789760789.719952:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.719958:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e070ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.719999:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e070ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.763357:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974149 is committed 00002000:00010000:1.0:1789760789.764024:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27975680 -> 27979776 00010000:00010000:1.0:1789760789.764046:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e070ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.764059:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e070ec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.764068:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e070ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.764075:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760789.764080:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e070ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.764090:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e070ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.764109:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e070ec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.764128:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e070ec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27975680->27979775] (req 27975680->27979775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.764229:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493860352:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44353us (44625us total) trans 4294974149 rc 0/0 00000100:00100000:1.0:1789760789.764240:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6860 00000100:00100000:1.0:1789760789.764261:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799997042511 time 800060892259 state 0x2 ref 0 00000100:00100000:2.0:1789760789.769943:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760789.770091:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493860480 00000100:00100000:1.0:1789760789.770101:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493860480 00000100:00100000:1.0:1789760789.770167:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 799997042511 time 800060892259 state 0x0 ref 1 00000100:00100000:1.0:1789760789.770183:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800061152675 time 800060892259 state 0x1 ref 1 00000100:00100000:1.0:1789760789.770214:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800061152675 time 800111249514 state 0x0 ref 1 00000100:00100000:1.0:1789760789.770230:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493860480:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760789.770283:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27979776, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:1.0:1789760789.770293:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070f3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.770310:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=800s v=(0 0) 00010000:00010000:1.0:1789760789.770315:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.770321:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.770336:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760789.812781:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27979776 -> 27983872 00010000:00010000:1.0:1789760789.812801:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.812813:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070f3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.812822:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.812829:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760789.812834:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.812842:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070f3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.812866:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070f3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.812874:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070f3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27979776->27983871] (req 27979776->27983871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.813002:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493860480:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42770us (43064us total) trans 4294974150 rc 0/0 00000100:00100000:1.0:1789760789.813013:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6861 00000100:00100000:1.0:1789760789.813024:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800061152675 time 800111249514 state 0x2 ref 0 00000001:00080000:3.0:1789760789.813114:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974150 is committed 00000100:00100000:2.0:1789760789.818410:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760789.822350:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493860608 00000100:00100000:1.0:1789760789.822364:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493860608 00000100:00100000:1.0:1789760789.822428:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800061152675 time 800111249514 state 0x0 ref 1 00000100:00100000:1.0:1789760789.822445:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800111509930 time 800111249514 state 0x1 ref 1 00000100:00100000:1.0:1789760789.822483:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800111509930 time 800163518048 state 0x0 ref 1 00000100:00100000:1.0:1789760789.822497:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493860608:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760789.822551:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27983872, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:1.0:1789760789.822560:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070fa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.822577:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=800s v=(0 0) 00010000:00010000:1.0:1789760789.822584:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.822591:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.822605:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.880179:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974151 is committed 00002000:00010000:1.0:1789760789.882934:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27983872 -> 27987968 00010000:00010000:1.0:1789760789.882957:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.883003:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070fa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.883013:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.883021:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760789.883027:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.883037:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070fa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.883056:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070fa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.883064:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e070fa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27983872->27987967] (req 27983872->27987967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.883181:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493860608:12345-192.168.203.60@tcp:4:dd.0 Request processed in 60673us (64767us total) trans 4294974151 rc 0/0 00000100:00100000:1.0:1789760789.883199:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6862 00000100:00100000:1.0:1789760789.883209:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800111509930 time 800163518048 state 0x2 ref 0 00000100:00100000:2.0:1789760789.883499:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc33ece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3303:1876699519262080:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.883551:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760789.883617:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1789760789.883623:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620ffc15500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3303:1876699519261696:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.883657:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760789.883680:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc33ed500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3303:1876699519262208:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.883702:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760789.883728:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620fadb5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3305:1876699519261824:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.883751:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760789.883769:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc33ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519262336:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.883791:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760789.883810:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789760789.883815:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc33ed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519261952:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.883837:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760789.883868:0:10519:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519262080 00000100:00100000:2.0:1789760789.883874:0:10519:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519262080 00000100:00100000:2.0:1789760789.883920:0:10519:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519262208 00000100:00100000:2.0:1789760789.883923:0:10519:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519262208 00000100:00100000:2.0:1789760789.883945:0:10519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 529 00000100:00100000:2.0:1789760789.883954:0:10519:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876699519262080:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.884027:0:10519:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876699519262080:12345-0@lo:400:kworker.0 Request processed in 75us (476us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760789.884035:0:10519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 529 00000100:00100000:2.0:1789760789.884047:0:10519:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519261824 00000100:00100000:2.0:1789760789.884050:0:10519:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519261824 00000100:00100000:2.0:1789760789.884066:0:10519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 530 00000100:00100000:2.0:1789760789.884071:0:10519:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876699519262208:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.884100:0:10519:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876699519262208:12345-0@lo:400:kworker.0 Request processed in 29us (397us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760789.884105:0:10519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 530 00000100:00100000:2.0:1789760789.884127:0:10519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 531 00000100:00100000:2.0:1789760789.884134:0:10519:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3305:x1876699519261824:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.884165:0:10519:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3305:x1876699519261824:12345-0@lo:400:kworker.0 Request processed in 32us (413us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760789.885884:0:10519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 531 00000100:00100000:2.0:1789760789.885921:0:5839:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519261696 00000100:00100000:2.0:1789760789.885928:0:5839:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519261696 00000100:00100000:2.0:1789760789.885981:0:5839:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 418 00000100:00100000:2.0:1789760789.885991:0:5839:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d04850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3303:x1876699519261696:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.886056:0:5839:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d04850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3303:x1876699519261696:12345-0@lo:400:kworker.0 Request processed in 67us (2398us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760789.886062:0:5839:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 418 00000100:00100000:2.0:1789760789.886098:0:13040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519262336 00000100:00100000:2.0:1789760789.886102:0:13040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519262336 00000100:00100000:2.0:1789760789.886141:0:13040:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519261952 00000100:00100000:2.0:1789760789.886145:0:13040:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519261952 00000100:00100000:2.0:1789760789.886168:0:13040:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 188 00000100:00100000:2.0:1789760789.886402:0:13040:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519262336:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.886442:0:13040:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519262336:12345-0@lo:400:kworker.0 Request processed in 43us (2649us total) trans 0 rc 0/0 00000100:00080000:1.0:1789760789.887249:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789760789.887255:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760789.887621:0:6689:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 189 00000100:00100000:0.0:1789760789.887640:0:6689:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519261952:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760789.887698:0:6689:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519261952:12345-0@lo:400:kworker.0 Request processed in 67us (3860us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760789.887705:0:6689:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 189 00000100:00100000:2.0:1789760789.887784:0:13040:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 188 00000100:00100000:2.0:1789760789.887839:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc33ed180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519261952:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.887854:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620fadb5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3305:1876699519261824:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.887866:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc33ef800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519262336:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.887907:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620ffc15500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3303:1876699519261696:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.887920:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc33ed500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3303:1876699519262208:0@lo:400:kworker.0 00000100:00100000:2.0:1789760789.887931:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc33ece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3303:1876699519262080:0@lo:400:kworker.0 00000100:00100000:0.0:1789760789.890182:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760789.892196:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493860736 00000100:00100000:3.0:1789760789.892209:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493860736 00000100:00100000:3.0:1789760789.892684:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800111509930 time 800163518048 state 0x0 ref 1 00000100:00100000:3.0:1789760789.892704:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800163778464 time 800163518048 state 0x1 ref 1 00000100:00100000:3.0:1789760789.892751:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800163778464 time 800233786526 state 0x0 ref 1 00000100:00100000:3.0:1789760789.892767:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493860736:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760789.892818:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27987968, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:3.0:1789760789.892827:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e07101 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.892844:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=800s v=(0 0) 00010000:00010000:3.0:1789760789.892849:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.892855:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e07101 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.892868:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e07101 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760789.937020:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27987968 -> 27992064 00000001:00080000:2.0:1789760789.937020:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974152 is committed 00010000:00010000:1.0:1789760789.937038:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e07101 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.937051:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e07101 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.937060:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e07101 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.937067:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760789.937072:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e07101 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.937087:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e07101 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.937108:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e07101 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.937132:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e07101 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27987968->27992063] (req 27987968->27992063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.937320:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493860736:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44552us (47150us total) trans 4294974152 rc 0/0 00000100:00100000:1.0:1789760789.937331:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6863 00000100:00100000:1.0:1789760789.937340:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800163778464 time 800233786526 state 0x2 ref 0 00000100:00100000:0.0:1789760789.941401:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760789.941572:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493860864 00000100:00100000:3.0:1789760789.941589:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493860864 00000100:00100000:3.0:1789760789.942342:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800163778464 time 800233786526 state 0x0 ref 1 00000100:00100000:3.0:1789760789.942371:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800234046942 time 800233786526 state 0x1 ref 1 00000100:00100000:3.0:1789760789.942411:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800234046942 time 800283445835 state 0x0 ref 1 00000100:00100000:3.0:1789760789.942430:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493860864:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760789.942501:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27992064, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:3.0:1789760789.942512:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07108 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.942533:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=800s v=(0 0) 00010000:00010000:3.0:1789760789.942539:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.942545:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07108 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.942563:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07108 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.976058:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974153 is committed 00002000:00010000:0.0:1789760789.976285:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27992064 -> 27996160 00010000:00010000:0.0:1789760789.976300:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07108 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.976313:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07108 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.976322:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07108 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.976330:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760789.976335:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07108 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.976344:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07108 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.976363:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07108 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.976371:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07108 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27992064->27996159] (req 27992064->27996159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.976454:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493860864:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34026us (35058us total) trans 4294974153 rc 0/0 00000100:00100000:0.0:1789760789.976465:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6864 00000100:00100000:0.0:1789760789.976474:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800234046942 time 800283445835 state 0x2 ref 0 00000100:00100000:2.0:1789760789.980055:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760789.980160:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493860992 00000100:00100000:1.0:1789760789.980179:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493860992 00000100:00100000:1.0:1789760789.980235:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800234046942 time 800283445835 state 0x0 ref 1 00000100:00100000:1.0:1789760789.980250:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800283706251 time 800283445835 state 0x1 ref 1 00000100:00100000:1.0:1789760789.980281:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800283706251 time 800321317138 state 0x0 ref 1 00000100:00100000:1.0:1789760789.980293:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493860992:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760789.980342:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27996160, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:1.0:1789760789.980350:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0710f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.980364:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=800s v=(0 0) 00010000:00010000:1.0:1789760789.980369:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.980375:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0710f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.980393:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0710f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.017618:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974154 is committed 00002000:00010000:1.0:1789760790.017848:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27996160 -> 28000256 00010000:00010000:1.0:1789760790.017863:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0710f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.017878:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0710f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.017886:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0710f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.017894:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760790.017899:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0710f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.017908:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0710f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.017928:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0710f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.017936:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0710f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27996160->28000255] (req 27996160->28000255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.018130:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493860992:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37825us (38071us total) trans 4294974154 rc 0/0 00000100:00100000:1.0:1789760790.018142:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6865 00000100:00100000:1.0:1789760790.018152:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800283706251 time 800321317138 state 0x2 ref 0 00000100:00100000:2.0:1789760790.023027:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760790.023151:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493861120 00000100:00100000:3.0:1789760790.023162:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493861120 00000100:00100000:3.0:1789760790.023227:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800283706251 time 800321317138 state 0x0 ref 1 00000100:00100000:3.0:1789760790.023242:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800321577554 time 800321317138 state 0x1 ref 1 00000100:00100000:3.0:1789760790.023273:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800321577554 time 800364308696 state 0x0 ref 1 00000100:00100000:3.0:1789760790.023286:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb17050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493861120:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760790.023338:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28000256, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:3.0:1789760790.023349:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07116 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.023364:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=800s v=(0 0) 00010000:00010000:3.0:1789760790.023368:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.023374:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07116 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.023388:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07116 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.064480:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974155 is committed 00002000:00010000:3.0:1789760790.064629:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28000256 -> 28004352 00010000:00010000:3.0:1789760790.064643:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07116 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.064656:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07116 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.064665:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07116 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.064685:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760790.064690:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07116 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.064700:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07116 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.064722:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07116 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.064731:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07116 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28000256->28004351] (req 28000256->28004351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.064826:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb17050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493861120:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41538us (41805us total) trans 4294974155 rc 0/0 00000100:00100000:3.0:1789760790.064836:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6866 00000100:00100000:3.0:1789760790.064845:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800321577554 time 800364308696 state 0x2 ref 0 00000100:00100000:1.0:1789760790.069327:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760790.069400:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493861248 00000100:00100000:3.0:1789760790.069411:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493861248 00000100:00100000:3.0:1789760790.069465:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800321577554 time 800364308696 state 0x0 ref 1 00000100:00100000:3.0:1789760790.069481:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800364569112 time 800364308696 state 0x1 ref 1 00000100:00100000:3.0:1789760790.069510:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800364569112 time 800410545408 state 0x0 ref 1 00000100:00100000:3.0:1789760790.069524:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493861248:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760790.069571:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28004352, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:3.0:1789760790.069580:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0711d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.069596:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=800s v=(0 0) 00010000:00010000:3.0:1789760790.069601:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.069607:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0711d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.069621:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0711d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.074035:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760790.103252:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974156 is committed 00002000:00010000:1.0:1789760790.103578:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28004352 -> 28008448 00010000:00010000:1.0:1789760790.103594:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0711d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.103605:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0711d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.103612:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0711d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.103618:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760790.103623:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0711d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.103632:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0711d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.103650:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0711d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.103662:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e0711d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28004352->28008447] (req 28004352->28008447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.103857:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493861248:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34331us (34535us total) trans 4294974156 rc 0/0 00000100:00100000:1.0:1789760790.103868:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6867 00000100:00100000:1.0:1789760790.103877:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800364569112 time 800410545408 state 0x2 ref 0 00000100:00100000:0.0:1789760790.107300:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760790.107474:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493861376 00000100:00100000:3.0:1789760790.107483:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493861376 00000100:00100000:3.0:1789760790.107531:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800364569112 time 800410545408 state 0x0 ref 1 00000100:00100000:3.0:1789760790.107546:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800410805824 time 800410545408 state 0x1 ref 1 00000100:00100000:3.0:1789760790.107606:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800410805824 time 800448641574 state 0x0 ref 1 00000100:00100000:3.0:1789760790.107617:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493861376:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760790.107660:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28008448, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:3.0:1789760790.107677:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07124 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.107692:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=800s v=(0 0) 00010000:00010000:3.0:1789760790.107696:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.107702:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07124 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.107716:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07124 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760790.134026:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28008448 -> 28012544 00010000:00010000:1.0:1789760790.134046:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07124 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.134060:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07124 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.134069:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07124 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.134077:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760790.134082:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07124 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.134091:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07124 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.134140:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07124 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.134150:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07124 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28008448->28012543] (req 28008448->28012543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.134255:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493861376:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26633us (26959us total) trans 4294974157 rc 0/0 00000100:00100000:1.0:1789760790.134265:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6868 00000100:00100000:1.0:1789760790.134274:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800410805824 time 800448641574 state 0x2 ref 0 00000001:00080000:3.0:1789760790.134454:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974157 is committed 00000100:00100000:2.0:1789760790.138129:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760790.138203:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493861504 00000100:00100000:1.0:1789760790.138212:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493861504 00000100:00100000:1.0:1789760790.138268:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800410805824 time 800448641574 state 0x0 ref 1 00000100:00100000:1.0:1789760790.138284:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800448901990 time 800448641574 state 0x1 ref 1 00000100:00100000:1.0:1789760790.138318:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800448901990 time 800479353711 state 0x0 ref 1 00000100:00100000:1.0:1789760790.138330:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493861504:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760790.138380:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28012544, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:1.0:1789760790.138390:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0712b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.138407:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=800s v=(0 0) 00010000:00010000:1.0:1789760790.138412:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.138418:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0712b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.138432:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0712b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.175288:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974158 is committed 00002000:00010000:0.0:1789760790.175378:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28012544 -> 28016640 00010000:00010000:0.0:1789760790.175391:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0712b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.175400:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0712b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.175406:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0712b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.175411:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760790.175414:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0712b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.175420:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0712b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.175438:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0712b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.175443:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0712b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28012544->28016639] (req 28012544->28016639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.175518:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493861504:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37185us (37403us total) trans 4294974158 rc 0/0 00000100:00100000:0.0:1789760790.175528:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6869 00000100:00100000:0.0:1789760790.176321:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800448901990 time 800479353711 state 0x2 ref 0 00000100:00100000:2.0:1789760790.179182:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760790.179281:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493861632 00000100:00100000:0.0:1789760790.179291:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493861632 00000100:00100000:0.0:1789760790.179343:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800448901990 time 800479353711 state 0x0 ref 1 00000100:00100000:0.0:1789760790.179358:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800479614127 time 800479353711 state 0x1 ref 1 00000100:00100000:0.0:1789760790.179430:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800479614127 time 800520464193 state 0x0 ref 1 00000100:00100000:0.0:1789760790.179443:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493861632:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760790.179492:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28016640, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:0.0:1789760790.179500:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e07132 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.179516:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=800s v=(0 0) 00010000:00010000:0.0:1789760790.179520:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.179527:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e07132 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.179541:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e07132 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760790.208864:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28016640 -> 28020736 00010000:00010000:3.0:1789760790.208886:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e07132 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.208900:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e07132 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.208909:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e07132 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.208916:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760790.208921:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e07132 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.208931:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e07132 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.208951:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e07132 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.208988:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e07132 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28016640->28020735] (req 28016640->28020735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.209060:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974159 is committed 00000100:00100000:3.0:1789760790.209077:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493861632:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29632us (29898us total) trans 4294974159 rc 0/0 00000100:00100000:3.0:1789760790.209088:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6870 00000100:00100000:3.0:1789760790.209264:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800479614127 time 800520464193 state 0x2 ref 0 00000100:00100000:2.0:1789760790.212017:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760790.212354:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493861760 00000100:00100000:3.0:1789760790.212364:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493861760 00000100:00100000:3.0:1789760790.212418:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800479614127 time 800520464193 state 0x0 ref 1 00000100:00100000:3.0:1789760790.212432:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800520724609 time 800520464193 state 0x1 ref 1 00000100:00100000:3.0:1789760790.212463:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800520724609 time 800553498993 state 0x0 ref 1 00000100:00100000:3.0:1789760790.212475:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493861760:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760790.212522:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28020736, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:3.0:1789760790.212530:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07139 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.212544:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=800s v=(0 0) 00010000:00010000:3.0:1789760790.212549:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.212555:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07139 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.212569:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07139 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.247392:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974160 is committed 00002000:00010000:3.0:1789760790.247501:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28020736 -> 28024832 00010000:00010000:3.0:1789760790.247516:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07139 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.247530:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07139 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.247540:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07139 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.247547:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760790.247552:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07139 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.247561:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07139 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.247581:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07139 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.247590:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07139 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28020736->28024831] (req 28020736->28024831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.247805:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493861760:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35326us (35791us total) trans 4294974160 rc 0/0 00000100:00100000:3.0:1789760790.247816:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6871 00000100:00100000:3.0:1789760790.247826:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800520724609 time 800553498993 state 0x2 ref 0 00000100:00100000:2.0:1789760790.252402:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760790.252504:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493861888 00000100:00100000:1.0:1789760790.252515:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493861888 00000100:00100000:1.0:1789760790.252577:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800520724609 time 800553498993 state 0x0 ref 1 00000100:00100000:1.0:1789760790.252593:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800553759409 time 800553498993 state 0x1 ref 1 00000100:00100000:1.0:1789760790.252617:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800553759409 time 800593653210 state 0x0 ref 1 00000100:00100000:1.0:1789760790.252631:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493861888:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760790.252682:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28024832, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:1.0:1789760790.252691:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07140 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.252707:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=800s v=(0 0) 00010000:00010000:1.0:1789760790.252712:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.252717:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07140 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.252731:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07140 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760790.318817:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28024832 -> 28028928 00010000:00010000:1.0:1789760790.318841:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07140 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.318855:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07140 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.318864:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07140 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.318872:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760790.318878:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07140 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.318888:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07140 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.318910:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07140 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.318918:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07140 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28024832->28028927] (req 28024832->28028927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.319061:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493861888:12345-192.168.203.60@tcp:4:dd.0 Request processed in 66427us (66664us total) trans 4294974161 rc 0/0 00000100:00100000:1.0:1789760790.319075:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6872 00000100:00100000:1.0:1789760790.319085:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800553759409 time 800593653210 state 0x2 ref 0 00000001:00080000:0.0:1789760790.319469:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974161 is committed 00000100:00100000:3.0:1789760790.325122:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760790.325263:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493862016 00000100:00100000:0.0:1789760790.325273:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493862016 00000100:00100000:0.0:1789760790.325330:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800553759409 time 800593653210 state 0x0 ref 1 00000100:00100000:0.0:1789760790.325345:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800593913626 time 800593653210 state 0x1 ref 1 00000100:00100000:0.0:1789760790.325380:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800593913626 time 800666415822 state 0x0 ref 1 00000100:00100000:0.0:1789760790.325392:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3663480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493862016:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760790.325451:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28028928, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:0.0:1789760790.325460:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07147 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.325475:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=800s v=(0 0) 00010000:00010000:0.0:1789760790.325480:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.325486:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07147 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.325499:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07147 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.374338:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974162 is committed 00002000:00010000:0.0:1789760790.374418:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28028928 -> 28033024 00010000:00010000:0.0:1789760790.374433:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07147 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.374445:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07147 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.374454:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07147 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.374460:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760790.374464:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07147 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.374472:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07147 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.374491:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07147 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.374498:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07147 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28028928->28033023] (req 28028928->28033023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.374675:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3663480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493862016:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49279us (49566us total) trans 4294974162 rc 0/0 00000100:00100000:0.0:1789760790.374688:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6873 00000100:00100000:0.0:1789760790.374697:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800593913626 time 800666415822 state 0x2 ref 0 00000100:00100000:1.0:1789760790.380636:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760790.380775:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493862144 00000100:00100000:3.0:1789760790.380785:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493862144 00000100:00100000:3.0:1789760790.380840:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800593913626 time 800666415822 state 0x0 ref 1 00000100:00100000:3.0:1789760790.380855:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800666676238 time 800666415822 state 0x1 ref 1 00000100:00100000:3.0:1789760790.380885:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800666676238 time 800721920245 state 0x0 ref 1 00000100:00100000:3.0:1789760790.380897:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493862144:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760790.380944:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28033024, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:3.0:1789760790.380954:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0714e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.381005:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=800s v=(0 0) 00010000:00010000:3.0:1789760790.381010:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.381017:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0714e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.381030:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0714e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.394062:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789760790.394086:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620ffc14000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519262464:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760790.394160:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760790.394410:0:6691:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519262464 00000100:00100000:2.0:1789760790.394418:0:6691:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519262464 00000100:00100000:2.0:1789760790.394466:0:6691:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 219 00000100:00100000:2.0:1789760790.394474:0:6691:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519262464:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760790.394506:0:6691:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760790.394550:0:6691:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519262464:12345-0@lo:13:osp-pre-1-0.0 Request processed in 77us (390us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760790.394558:0:6691:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 219 00000100:00100000:2.0:1789760790.394709:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620ffc14000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519262464:0@lo:13:osp-pre-1-0.0 00000001:00080000:1.0:1789760790.430073:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974163 is committed 00002000:00010000:3.0:1789760790.430246:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28033024 -> 28037120 00010000:00010000:3.0:1789760790.430262:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0714e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.430274:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0714e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.430288:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0714e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.430295:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760790.430299:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0714e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.430308:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0714e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.430327:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0714e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.430335:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0714e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28033024->28037119] (req 28033024->28037119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.430432:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493862144:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49530us (49799us total) trans 4294974163 rc 0/0 00000100:00100000:3.0:1789760790.430442:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6874 00000100:00100000:3.0:1789760790.430452:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800666676238 time 800721920245 state 0x2 ref 0 00000100:00100000:0.0:1789760790.434346:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760790.434499:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493862272 00000100:00100000:1.0:1789760790.434510:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493862272 00000100:00100000:1.0:1789760790.434581:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800666676238 time 800721920245 state 0x0 ref 1 00000100:00100000:1.0:1789760790.434597:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800722180661 time 800721920245 state 0x1 ref 1 00000100:00100000:1.0:1789760790.434627:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800722180661 time 800775662343 state 0x0 ref 1 00000100:00100000:1.0:1789760790.434875:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493862272:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760790.434923:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28037120, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:1.0:1789760790.434932:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07155 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.434947:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=800s v=(0 0) 00010000:00010000:1.0:1789760790.434952:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.434958:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07155 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.435007:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07155 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.487791:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974164 is committed 00002000:00010000:2.0:1789760790.488029:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28037120 -> 28041216 00010000:00010000:2.0:1789760790.488047:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07155 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.488061:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07155 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.488071:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07155 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.488079:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760790.488084:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07155 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.488094:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07155 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.488123:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07155 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.488131:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07155 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28037120->28041215] (req 28037120->28041215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.488220:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493862272:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53345us (53877us total) trans 4294974164 rc 0/0 00000100:00100000:2.0:1789760790.488230:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6875 00000100:00100000:2.0:1789760790.488240:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800722180661 time 800775662343 state 0x2 ref 0 00000100:00100000:1.0:1789760790.494732:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760790.495029:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493862400 00000100:00100000:3.0:1789760790.495041:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493862400 00000100:00100000:2.0:1789760790.495203:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800722180661 time 800775662343 state 0x0 ref 1 00000100:00100000:2.0:1789760790.495229:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800775922759 time 800775662343 state 0x1 ref 1 00000100:00100000:2.0:1789760790.495259:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800775922759 time 800836294117 state 0x0 ref 1 00000100:00100000:2.0:1789760790.495273:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493862400:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760790.495322:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28041216, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:2.0:1789760790.495330:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0715c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.495346:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=800s v=(0 0) 00010000:00010000:2.0:1789760790.495351:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.495357:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0715c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.495371:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0715c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.548562:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974165 is committed 00002000:00010000:0.0:1789760790.548779:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28041216 -> 28045312 00010000:00010000:0.0:1789760790.548796:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0715c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.548809:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0715c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.548817:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0715c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.548824:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760790.548828:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0715c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.548837:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0715c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.548855:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0715c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.548862:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e0715c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28041216->28045311] (req 28041216->28045311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.549027:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493862400:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53751us (54300us total) trans 4294974165 rc 0/0 00000100:00100000:0.0:1789760790.549038:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6876 00000100:00100000:0.0:1789760790.549047:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800775922759 time 800836294117 state 0x2 ref 0 00000100:00100000:1.0:1789760790.558935:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760790.559211:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493862528 00000100:00100000:0.0:1789760790.559283:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493862528 00000100:00100000:0.0:1789760790.559340:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800775922759 time 800836294117 state 0x0 ref 1 00000100:00100000:0.0:1789760790.559355:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800836554533 time 800836294117 state 0x1 ref 1 00000100:00100000:0.0:1789760790.559383:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800836554533 time 800900418427 state 0x0 ref 1 00000100:00100000:0.0:1789760790.559396:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a34e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493862528:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760790.559451:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28045312, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:0.0:1789760790.559460:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07163 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.559477:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=800s v=(0 0) 00010000:00010000:0.0:1789760790.559481:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.559487:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07163 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.559500:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07163 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.613486:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974166 is committed 00002000:00010000:0.0:1789760790.613642:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28045312 -> 28049408 00010000:00010000:0.0:1789760790.613657:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07163 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.613668:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07163 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.613673:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07163 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.613677:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760790.613681:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07163 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.613687:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07163 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.613702:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07163 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.613706:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07163 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28045312->28049407] (req 28045312->28049407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.613781:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a34e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493862528:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54384us (54854us total) trans 4294974166 rc 0/0 00000100:00100000:0.0:1789760790.613790:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6877 00000100:00100000:0.0:1789760790.613797:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800836554533 time 800900418427 state 0x2 ref 0 00000100:00100000:2.0:1789760790.617702:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760790.618325:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493862656 00000100:00100000:3.0:1789760790.618337:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493862656 00000100:00100000:2.0:1789760790.618473:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800836554533 time 800900418427 state 0x0 ref 1 00000100:00100000:2.0:1789760790.618490:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800900678843 time 800900418427 state 0x1 ref 1 00000100:00100000:2.0:1789760790.618555:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800900678843 time 800959589100 state 0x0 ref 1 00000100:00100000:2.0:1789760790.618568:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493862656:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760790.618630:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28049408, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xd809 00010000:00010000:2.0:1789760790.618638:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0716a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.618654:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=800s v=(0 0) 00010000:00010000:2.0:1789760790.618659:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.618665:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0716a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.618680:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0716a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.677380:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974167 is committed 00002000:00010000:0.0:1789760790.677720:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28049408 -> 28053504 00002000:00010000:0.0:1789760790.677725:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760789 -> 1789760790 00002000:00010000:0.0:1789760790.677728:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760789 -> 1789760790 00010000:00010000:0.0:1789760790.677741:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0716a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.677754:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0716a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.677764:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0716a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.677772:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760790.677776:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0716a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.677786:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0716a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.677807:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0716a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.677815:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e0716a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28049408->28053503] (req 28049408->28053503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.677914:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493862656:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59343us (60216us total) trans 4294974167 rc 0/0 00000100:00100000:0.0:1789760790.677924:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6878 00000100:00100000:0.0:1789760790.677934:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800900678843 time 800959589100 state 0x2 ref 0 00000100:00100000:1.0:1789760790.682479:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760790.682767:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493862784 00000100:00100000:2.0:1789760790.682777:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493862784 00000100:00100000:2.0:1789760790.682829:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800900678843 time 800959589100 state 0x0 ref 1 00000100:00100000:2.0:1789760790.682843:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800959849516 time 800959589100 state 0x1 ref 1 00000100:00100000:2.0:1789760790.682957:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800959849516 time 801023990943 state 0x0 ref 1 00000100:00100000:2.0:1789760790.682987:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a37480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493862784:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760790.683037:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28053504, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:2.0:1789760790.683046:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07171 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.683061:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=801s v=(0 0) 00010000:00010000:2.0:1789760790.683066:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.683072:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07171 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.683085:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07171 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.718648:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974168 is committed 00002000:00010000:3.0:1789760790.718740:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28053504 -> 28057600 00010000:00010000:3.0:1789760790.718754:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07171 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.718766:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07171 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.718775:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07171 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.718783:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760790.718788:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07171 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.718798:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07171 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.718814:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07171 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.718822:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07171 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28053504->28057599] (req 28053504->28057599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.718905:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a37480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493862784:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35915us (36431us total) trans 4294974168 rc 0/0 00000100:00100000:3.0:1789760790.718914:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6879 00000100:00100000:3.0:1789760790.718923:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800959849516 time 801023990943 state 0x2 ref 0 00000100:00100000:2.0:1789760790.727758:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760790.727866:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493862912 00000100:00100000:0.0:1789760790.727876:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493862912 00000100:00100000:0.0:1789760790.727928:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 800959849516 time 801023990943 state 0x0 ref 1 00000100:00100000:0.0:1789760790.727943:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801024251359 time 801023990943 state 0x1 ref 1 00000100:00100000:0.0:1789760790.728013:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801024251359 time 801069048344 state 0x0 ref 1 00000100:00100000:0.0:1789760790.728027:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fadb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493862912:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760790.728074:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28057600, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:0.0:1789760790.728082:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07178 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.728097:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=801s v=(0 0) 00010000:00010000:0.0:1789760790.728102:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.728108:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07178 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.728132:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07178 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760790.769181:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28057600 -> 28061696 00010000:00010000:2.0:1789760790.769199:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07178 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.769213:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07178 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.769222:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07178 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.769229:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbe00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760790.769234:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07178 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.769249:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07178 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.769266:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07178 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.769273:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056d5b35f/0x3eb3dcc0b2e07178 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28057600->28061695] (req 28057600->28061695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.769353:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fadb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493862912:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41323us (41598us total) trans 4294974169 rc 0/0 00000100:00100000:2.0:1789760790.769364:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6880 00000100:00100000:2.0:1789760790.769373:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801024251359 time 801069048344 state 0x2 ref 0 00000001:00080000:1.0:1789760790.769443:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974169 is committed 00000100:00100000:2.0:1789760790.772077:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760790.772159:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493863040 00000100:00100000:1.0:1789760790.772168:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493863040 00000100:00100000:1.0:1789760790.772216:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801024251359 time 801069048344 state 0x0 ref 1 00000100:00100000:1.0:1789760790.772229:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801069308760 time 801069048344 state 0x1 ref 1 00000100:00100000:1.0:1789760790.772260:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801069308760 time 801113295435 state 0x0 ref 1 00000100:00100000:1.0:1789760790.772271:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4361500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493863040:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760790.772314:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28061696, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:1.0:1789760790.772323:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0717f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.772337:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=801s v=(0 0) 00010000:00010000:1.0:1789760790.772341:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.772347:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0717f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.772361:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0717f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.812499:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974170 is committed 00002000:00010000:1.0:1789760790.812641:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28061696 -> 28065792 00010000:00010000:1.0:1789760790.812657:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0717f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.812671:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0717f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.812681:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0717f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.812688:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760790.812692:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0717f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.812702:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0717f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.812721:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0717f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.812729:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0717f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28061696->28065791] (req 28061696->28065791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.812819:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4361500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493863040:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40544us (40745us total) trans 4294974170 rc 0/0 00000100:00100000:1.0:1789760790.812833:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6881 00000100:00100000:1.0:1789760790.812843:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801069308760 time 801113295435 state 0x2 ref 0 00000100:00100000:3.0:1789760790.815998:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760790.816251:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493863168 00000100:00100000:0.0:1789760790.816260:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493863168 00000100:00100000:0.0:1789760790.816308:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801069308760 time 801113295435 state 0x0 ref 1 00000100:00100000:0.0:1789760790.816323:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801113555851 time 801113295435 state 0x1 ref 1 00000100:00100000:0.0:1789760790.816348:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801113555851 time 801157384074 state 0x0 ref 1 00000100:00100000:0.0:1789760790.816358:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3662d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493863168:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760790.816401:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28065792, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:0.0:1789760790.816409:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e07186 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.816424:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=801s v=(0 0) 00010000:00010000:0.0:1789760790.816428:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.816434:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e07186 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.816447:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e07186 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.852955:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974171 is committed 00002000:00010000:0.0:1789760790.853231:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28065792 -> 28069888 00010000:00010000:0.0:1789760790.853246:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e07186 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.853263:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e07186 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.853272:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e07186 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.853278:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760790.853282:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e07186 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.853291:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e07186 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.853310:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e07186 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.853318:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e07186 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28065792->28069887] (req 28065792->28069887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.853438:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3662d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493863168:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37076us (37481us total) trans 4294974171 rc 0/0 00000100:00100000:0.0:1789760790.853449:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6882 00000100:00100000:0.0:1789760790.853458:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801113555851 time 801157384074 state 0x2 ref 0 00000100:00100000:3.0:1789760790.856670:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760790.856753:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493863296 00000100:00100000:2.0:1789760790.856762:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493863296 00000100:00100000:2.0:1789760790.856812:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801113555851 time 801157384074 state 0x0 ref 1 00000100:00100000:2.0:1789760790.856827:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801157644490 time 801157384074 state 0x1 ref 1 00000100:00100000:2.0:1789760790.856854:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801157644490 time 801197890569 state 0x0 ref 1 00000100:00100000:2.0:1789760790.856865:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3662680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493863296:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760790.856907:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28069888, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:2.0:1789760790.856916:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0718d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.856930:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=801s v=(0 0) 00010000:00010000:2.0:1789760790.856934:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.856940:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0718d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.856955:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0718d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.894627:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974172 is committed 00002000:00010000:3.0:1789760790.895404:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28069888 -> 28073984 00010000:00010000:3.0:1789760790.895419:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0718d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.895432:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0718d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.895441:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0718d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.895448:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760790.895453:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0718d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.895462:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0718d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.895481:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0718d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.895488:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e0718d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28069888->28073983] (req 28069888->28073983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.895580:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3662680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493863296:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38711us (38915us total) trans 4294974172 rc 0/0 00000100:00100000:3.0:1789760790.895644:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6883 00000100:00100000:3.0:1789760790.895654:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801157644490 time 801197890569 state 0x2 ref 0 00000100:00100000:0.0:1789760790.898945:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760790.899085:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493863424 00000100:00100000:1.0:1789760790.899094:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493863424 00000100:00100000:1.0:1789760790.899157:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801157644490 time 801197890569 state 0x0 ref 1 00000100:00100000:1.0:1789760790.899233:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801198150985 time 801197890569 state 0x1 ref 1 00000100:00100000:1.0:1789760790.899289:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801198150985 time 801240324978 state 0x0 ref 1 00000100:00100000:1.0:1789760790.899302:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493863424:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760790.899349:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28073984, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:1.0:1789760790.899358:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07194 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.899373:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=801s v=(0 0) 00010000:00010000:1.0:1789760790.899377:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.899384:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07194 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.899397:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07194 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.925349:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974173 is committed 00002000:00010000:1.0:1789760790.925524:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28073984 -> 28078080 00010000:00010000:1.0:1789760790.925541:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07194 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.925555:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07194 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.925565:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07194 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.925574:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760790.925579:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07194 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.925600:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07194 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.925621:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07194 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.925629:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07194 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28073984->28078079] (req 28073984->28078079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.925729:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493863424:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26423us (26787us total) trans 4294974173 rc 0/0 00000100:00100000:1.0:1789760790.925740:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6884 00000100:00100000:1.0:1789760790.925750:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801198150985 time 801240324978 state 0x2 ref 0 00000100:00100000:1.0:1789760790.930036:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760790.930158:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493863552 00000100:00100000:2.0:1789760790.930172:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493863552 00000100:00100000:2.0:1789760790.930225:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801198150985 time 801240324978 state 0x0 ref 1 00000100:00100000:2.0:1789760790.930240:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801240585394 time 801240324978 state 0x1 ref 1 00000100:00100000:2.0:1789760790.930262:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801240585394 time 801271297759 state 0x0 ref 1 00000100:00100000:2.0:1789760790.930274:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493863552:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760790.930321:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28078080, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:2.0:1789760790.930331:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0719b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.930347:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=801s v=(0 0) 00010000:00010000:2.0:1789760790.930351:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.930357:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0719b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.930371:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0719b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760790.966906:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28078080 -> 28082176 00000001:00080000:3.0:1789760790.966921:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974174 is committed 00010000:00010000:2.0:1789760790.966926:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0719b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.966938:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0719b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.966947:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0719b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.966955:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760790.966975:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0719b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.966986:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0719b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.967004:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0719b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.967012:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0719b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28078080->28082175] (req 28078080->28082175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.967284:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493863552:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37006us (37251us total) trans 4294974174 rc 0/0 00000100:00100000:2.0:1789760790.967295:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6885 00000100:00100000:2.0:1789760790.967304:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801240585394 time 801271297759 state 0x2 ref 0 00000100:00100000:2.0:1789760790.969517:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760790.969619:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493863680 00000100:00100000:0.0:1789760790.969629:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493863680 00000100:00100000:0.0:1789760790.969681:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801240585394 time 801271297759 state 0x0 ref 1 00000100:00100000:0.0:1789760790.969697:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801271558175 time 801271297759 state 0x1 ref 1 00000100:00100000:0.0:1789760790.969735:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801271558175 time 801310770946 state 0x0 ref 1 00000100:00100000:0.0:1789760790.969748:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4363480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493863680:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760790.969796:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28082176, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:0.0:1789760790.969806:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e071a2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.969822:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=801s v=(0 0) 00010000:00010000:0.0:1789760790.969827:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.969833:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e071a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.969846:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e071a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.995194:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974175 is committed 00002000:00010000:0.0:1789760790.995250:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28082176 -> 28086272 00010000:00010000:0.0:1789760790.995264:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e071a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.995283:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e071a2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.995292:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e071a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.995299:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760790.995304:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e071a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.995313:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e071a2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.995332:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e071a2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.995339:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e071a2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28082176->28086271] (req 28082176->28086271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.995428:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4363480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493863680:12345-192.168.203.60@tcp:4:dd.0 Request processed in 25678us (25915us total) trans 4294974175 rc 0/0 00000100:00100000:0.0:1789760790.995439:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6886 00000100:00100000:0.0:1789760790.995451:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801271558175 time 801310770946 state 0x2 ref 0 00000100:00100000:0.0:1789760790.998982:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760790.999283:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493863808 00000100:00100000:3.0:1789760790.999293:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493863808 00000100:00100000:3.0:1789760790.999345:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801271558175 time 801310770946 state 0x0 ref 1 00000100:00100000:3.0:1789760790.999361:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801311031362 time 801310770946 state 0x1 ref 1 00000100:00100000:3.0:1789760790.999387:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801311031362 time 801340422162 state 0x0 ref 1 00000100:00100000:3.0:1789760790.999399:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493863808:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760790.999446:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28086272, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:3.0:1789760790.999455:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071a9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.999472:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=801s v=(0 0) 00010000:00010000:3.0:1789760790.999477:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.999483:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.999498:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.047821:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974176 is committed 00002000:00010000:3.0:1789760791.047950:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28086272 -> 28090368 00010000:00010000:3.0:1789760791.049124:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.049141:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071a9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.049150:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.049157:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760791.049170:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.049180:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071a9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.049208:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071a9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.049216:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071a9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28086272->28090367] (req 28086272->28090367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.049632:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493863808:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50228us (50669us total) trans 4294974176 rc 0/0 00000100:00100000:3.0:1789760791.049646:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6887 00000100:00100000:3.0:1789760791.049655:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801311031362 time 801340422162 state 0x2 ref 0 00000100:00100000:1.0:1789760791.059645:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760791.059754:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493863936 00000100:00100000:2.0:1789760791.059763:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493863936 00000100:00100000:2.0:1789760791.059837:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801311031362 time 801340422162 state 0x0 ref 1 00000100:00100000:2.0:1789760791.059852:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801340682578 time 801340422162 state 0x1 ref 1 00000100:00100000:2.0:1789760791.059886:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801340682578 time 801400921366 state 0x0 ref 1 00000100:00100000:2.0:1789760791.059900:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493863936:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760791.059948:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28090368, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:2.0:1789760791.059957:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e071b0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.060011:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=801s v=(0 0) 00010000:00010000:2.0:1789760791.060015:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.060021:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e071b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.060040:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e071b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760791.060404:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789760791.093049:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28090368 -> 28094464 00010000:00010000:2.0:1789760791.093067:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e071b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.093167:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e071b0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.093178:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e071b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.093185:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760791.093189:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e071b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.093199:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e071b0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.093216:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e071b0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.093224:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e071b0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28090368->28094463] (req 28090368->28094463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.093401:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493863936:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33500us (33761us total) trans 4294974177 rc 0/0 00000100:00100000:2.0:1789760791.093414:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6888 00000100:00100000:2.0:1789760791.093424:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801340682578 time 801400921366 state 0x2 ref 0 00000001:00080000:1.0:1789760791.093799:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974177 is committed 00010000:00010000:3.0:1789760791.098812:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789760791.099406:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760791.099475:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493864064 00000100:00100000:2.0:1789760791.099485:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493864064 00000100:00100000:2.0:1789760791.099539:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801340682578 time 801400921366 state 0x0 ref 1 00000100:00100000:2.0:1789760791.099553:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801401181782 time 801400921366 state 0x1 ref 1 00000100:00100000:2.0:1789760791.099585:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801401181782 time 801440619842 state 0x0 ref 1 00000100:00100000:2.0:1789760791.099598:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1d08e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493864064:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760791.099647:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28094464, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:2.0:1789760791.099655:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e071b7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.099671:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=801s v=(0 0) 00010000:00010000:2.0:1789760791.099675:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.099682:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e071b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.099695:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e071b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.146811:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974178 is committed 00002000:00010000:2.0:1789760791.146935:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28094464 -> 28098560 00010000:00010000:2.0:1789760791.146951:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e071b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.146984:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e071b7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.146993:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e071b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.147000:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760791.147005:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e071b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.147014:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e071b7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.147041:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e071b7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.147049:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e071b7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28094464->28098559] (req 28094464->28098559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.147151:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1d08e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493864064:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47551us (47746us total) trans 4294974178 rc 0/0 00000100:00100000:2.0:1789760791.147171:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6889 00000100:00100000:2.0:1789760791.147181:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801401181782 time 801440619842 state 0x2 ref 0 00000100:00100000:2.0:1789760791.152987:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760791.153037:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493864192 00000100:00100000:0.0:1789760791.153048:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493864192 00000100:00100000:0.0:1789760791.153104:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801401181782 time 801440619842 state 0x0 ref 1 00000100:00100000:0.0:1789760791.153134:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801440880258 time 801440619842 state 0x1 ref 1 00000100:00100000:0.0:1789760791.153163:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801440880258 time 801494198825 state 0x0 ref 1 00000100:00100000:0.0:1789760791.153637:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493864192:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760791.153687:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28098560, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:0.0:1789760791.153696:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e071be lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.153711:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=801s v=(0 0) 00010000:00010000:0.0:1789760791.153716:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.153723:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e071be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.153737:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e071be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.198599:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974179 is committed 00002000:00010000:1.0:1789760791.198761:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28098560 -> 28102656 00010000:00010000:1.0:1789760791.198776:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e071be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.198788:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e071be lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.198797:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e071be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.198804:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760791.198809:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e071be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.198818:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e071be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.198836:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e071be lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.198843:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e071be lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28098560->28102655] (req 28098560->28102655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.198942:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4362d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493864192:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45305us (45983us total) trans 4294974179 rc 0/0 00000100:00100000:1.0:1789760791.198953:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6890 00000100:00100000:1.0:1789760791.199002:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801440880258 time 801494198825 state 0x2 ref 0 00000100:00100000:1.0:1789760791.202254:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760791.202379:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493864320 00000100:00100000:3.0:1789760791.202389:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493864320 00000100:00100000:3.0:1789760791.202442:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801440880258 time 801494198825 state 0x0 ref 1 00000100:00100000:3.0:1789760791.202457:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801494459241 time 801494198825 state 0x1 ref 1 00000100:00100000:3.0:1789760791.202477:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801494459241 time 801543513535 state 0x0 ref 1 00000100:00100000:3.0:1789760791.202489:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c89fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493864320:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760791.202535:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28102656, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:3.0:1789760791.202544:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071c5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.202559:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=801s v=(0 0) 00010000:00010000:3.0:1789760791.202564:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.202570:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.202658:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760791.233916:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28102656 -> 28106752 00010000:00010000:3.0:1789760791.233935:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.233949:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071c5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.233973:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.233981:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760791.233987:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.233996:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071c5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.234015:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071c5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.234022:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071c5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28102656->28106751] (req 28102656->28106751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.234097:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c89fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493864320:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31605us (31848us total) trans 4294974180 rc 0/0 00000100:00100000:3.0:1789760791.234121:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6891 00000100:00100000:3.0:1789760791.234131:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801494459241 time 801543513535 state 0x2 ref 0 00000001:00080000:3.0:1789760791.235262:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974180 is committed 00000100:00100000:2.0:1789760791.237666:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760791.237777:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493864448 00000100:00100000:1.0:1789760791.237786:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493864448 00000100:00100000:1.0:1789760791.237842:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801494459241 time 801543513535 state 0x0 ref 1 00000100:00100000:1.0:1789760791.237858:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801543773951 time 801543513535 state 0x1 ref 1 00000100:00100000:1.0:1789760791.237935:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801543773951 time 801578969738 state 0x0 ref 1 00000100:00100000:1.0:1789760791.237949:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493864448:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760791.238036:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28106752, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:1.0:1789760791.238045:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e071cc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.238061:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=801s v=(0 0) 00010000:00010000:1.0:1789760791.238066:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.238073:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e071cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.238087:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e071cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760791.271355:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28106752 -> 28110848 00010000:00010000:2.0:1789760791.271374:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e071cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.271388:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e071cc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.271398:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e071cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.271406:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760791.271411:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e071cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.271421:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e071cc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.271440:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e071cc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.271448:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e071cc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28106752->28110847] (req 28106752->28110847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.271537:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4360700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493864448:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33587us (33875us total) trans 4294974181 rc 0/0 00000100:00100000:2.0:1789760791.271548:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6892 00000100:00100000:2.0:1789760791.271558:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801543773951 time 801578969738 state 0x2 ref 0 00000001:00080000:3.0:1789760791.271638:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974181 is committed 00000100:00100000:0.0:1789760791.274500:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760791.274741:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493864576 00000100:00100000:1.0:1789760791.274751:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493864576 00000100:00100000:1.0:1789760791.274801:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801543773951 time 801578969738 state 0x0 ref 1 00000100:00100000:1.0:1789760791.274816:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801579230154 time 801578969738 state 0x1 ref 1 00000100:00100000:1.0:1789760791.274847:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801579230154 time 801615883290 state 0x0 ref 1 00000100:00100000:1.0:1789760791.274860:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493864576:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760791.274906:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28110848, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:1.0:1789760791.274914:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e071d3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.274929:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=801s v=(0 0) 00010000:00010000:1.0:1789760791.274933:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.274940:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e071d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.274954:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e071d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.319798:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974182 is committed 00002000:00010000:1.0:1789760791.319819:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28110848 -> 28114944 00010000:00010000:1.0:1789760791.319833:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e071d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.319853:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e071d3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.319862:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e071d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.319870:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760791.319875:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e071d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.319884:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e071d3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.319904:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e071d3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.319911:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e071d3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28110848->28114943] (req 28110848->28114943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.320062:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493864576:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45189us (45557us total) trans 4294974182 rc 0/0 00000100:00100000:1.0:1789760791.320077:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6893 00000100:00100000:1.0:1789760791.320087:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801579230154 time 801615883290 state 0x2 ref 0 00000100:00100000:1.0:1789760791.323991:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760791.324099:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493864704 00000100:00100000:0.0:1789760791.324118:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493864704 00000100:00100000:0.0:1789760791.324184:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801579230154 time 801615883290 state 0x0 ref 1 00000100:00100000:0.0:1789760791.324199:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801616143706 time 801615883290 state 0x1 ref 1 00000100:00100000:0.0:1789760791.324228:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801616143706 time 801665263453 state 0x0 ref 1 00000100:00100000:0.0:1789760791.324241:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493864704:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760791.324288:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28114944, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:0.0:1789760791.324297:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e071da lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.324312:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=801s v=(0 0) 00010000:00010000:0.0:1789760791.324317:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.324323:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e071da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.324337:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e071da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760791.353434:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28114944 -> 28119040 00010000:00010000:2.0:1789760791.353457:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e071da lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.353471:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e071da lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.353481:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e071da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.353489:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760791.353494:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e071da lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.353503:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e071da lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.353523:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e071da lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.353531:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e071da lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28114944->28119039] (req 28114944->28119039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.353632:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493864704:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29388us (29678us total) trans 4294974183 rc 0/0 00000100:00100000:2.0:1789760791.353647:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6894 00000100:00100000:2.0:1789760791.353656:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801616143706 time 801665263453 state 0x2 ref 0 00000001:00080000:3.0:1789760791.353789:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974183 is committed 00000100:00100000:2.0:1789760791.358572:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760791.358686:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493864832 00000100:00100000:3.0:1789760791.358696:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493864832 00000100:00100000:3.0:1789760791.358752:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801616143706 time 801665263453 state 0x0 ref 1 00000100:00100000:3.0:1789760791.358769:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801665523869 time 801665263453 state 0x1 ref 1 00000100:00100000:3.0:1789760791.358801:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801665523869 time 801699836907 state 0x0 ref 1 00000100:00100000:3.0:1789760791.358815:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493864832:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760791.358863:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28119040, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:3.0:1789760791.358872:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071e1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.358887:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=801s v=(0 0) 00010000:00010000:3.0:1789760791.358891:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.358897:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.358911:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.382545:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974184 is committed 00002000:00010000:2.0:1789760791.382656:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28119040 -> 28123136 00010000:00010000:2.0:1789760791.382673:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.382685:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071e1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.382694:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.382701:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760791.382706:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.382716:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071e1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.382735:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071e1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.382742:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071e1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28119040->28123135] (req 28119040->28123135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.382839:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493864832:12345-192.168.203.60@tcp:4:dd.0 Request processed in 24021us (24272us total) trans 4294974184 rc 0/0 00000100:00100000:2.0:1789760791.382849:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6895 00000100:00100000:2.0:1789760791.382859:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801665523869 time 801699836907 state 0x2 ref 0 00000100:00100000:1.0:1789760791.386278:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760791.386744:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493864960 00000100:00100000:2.0:1789760791.386756:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493864960 00000100:00100000:2.0:1789760791.386813:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801665523869 time 801699836907 state 0x0 ref 1 00000100:00100000:2.0:1789760791.386830:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801700097323 time 801699836907 state 0x1 ref 1 00000100:00100000:2.0:1789760791.386895:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801700097323 time 801727929861 state 0x0 ref 1 00000100:00100000:2.0:1789760791.386908:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493864960:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760791.386983:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28123136, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:2.0:1789760791.386993:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e071e8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.387008:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=801s v=(0 0) 00010000:00010000:2.0:1789760791.387014:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.387020:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e071e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.387032:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e071e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.418381:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974185 is committed 00002000:00010000:2.0:1789760791.418483:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28123136 -> 28127232 00010000:00010000:2.0:1789760791.418498:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e071e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.418511:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e071e8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.418520:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e071e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.418527:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760791.418532:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e071e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.418541:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e071e8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.418560:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e071e8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.418567:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e071e8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28123136->28127231] (req 28123136->28127231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.418700:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493864960:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31788us (32426us total) trans 4294974185 rc 0/0 00000100:00100000:2.0:1789760791.418714:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6896 00000100:00100000:2.0:1789760791.418724:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801700097323 time 801727929861 state 0x2 ref 0 00000100:00100000:1.0:1789760791.425195:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760791.425510:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493865088 00000100:00100000:2.0:1789760791.425524:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493865088 00000100:00100000:2.0:1789760791.425685:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801700097323 time 801727929861 state 0x0 ref 1 00000100:00100000:2.0:1789760791.425708:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801728190277 time 801727929861 state 0x1 ref 1 00000100:00100000:2.0:1789760791.425794:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801728190277 time 801766828815 state 0x0 ref 1 00000100:00100000:2.0:1789760791.425809:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493865088:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760791.425863:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28127232, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:2.0:1789760791.425873:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e071ef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.425890:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=801s v=(0 0) 00010000:00010000:2.0:1789760791.425894:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.425900:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e071ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.425924:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e071ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760791.455827:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28127232 -> 28131328 00010000:00010000:2.0:1789760791.455846:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e071ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.455860:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e071ef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.455869:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e071ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.455876:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760791.455881:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e071ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.455891:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e071ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.455911:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e071ef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.455919:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e071ef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28127232->28131327] (req 28127232->28131327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.456065:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974186 is committed 00000100:00100000:2.0:1789760791.456168:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493865088:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30349us (30970us total) trans 4294974186 rc 0/0 00000100:00100000:2.0:1789760791.456180:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6897 00000100:00100000:2.0:1789760791.456191:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801728190277 time 801766828815 state 0x2 ref 0 00000100:00100000:1.0:1789760791.458890:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760791.458996:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493865216 00000100:00100000:2.0:1789760791.459008:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493865216 00000100:00100000:2.0:1789760791.459070:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801728190277 time 801766828815 state 0x0 ref 1 00000100:00100000:2.0:1789760791.459086:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801767089231 time 801766828815 state 0x1 ref 1 00000100:00100000:2.0:1789760791.459119:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801767089231 time 801800154930 state 0x0 ref 1 00000100:00100000:2.0:1789760791.459131:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493865216:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760791.459195:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28131328, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:2.0:1789760791.459204:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e071f6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.459220:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=801s v=(0 0) 00010000:00010000:2.0:1789760791.459224:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.459230:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e071f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.459244:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e071f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760791.481031:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28131328 -> 28135424 00010000:00010000:1.0:1789760791.481050:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e071f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.481064:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e071f6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.481074:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e071f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.481082:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760791.481087:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e071f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.481097:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e071f6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.481129:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e071f6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.481137:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e071f6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28131328->28135423] (req 28131328->28135423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.481243:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493865216:12345-192.168.203.60@tcp:4:dd.0 Request processed in 22108us (22357us total) trans 4294974187 rc 0/0 00000100:00100000:1.0:1789760791.481253:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6898 00000100:00100000:1.0:1789760791.481263:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801767089231 time 801800154930 state 0x2 ref 0 00000001:00080000:1.0:1789760791.481303:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974187 is committed 00000100:00100000:1.0:1789760791.485456:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760791.485545:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493865344 00000100:00100000:3.0:1789760791.485554:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493865344 00000100:00100000:3.0:1789760791.485609:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801767089231 time 801800154930 state 0x0 ref 1 00000100:00100000:3.0:1789760791.485624:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801800415346 time 801800154930 state 0x1 ref 1 00000100:00100000:3.0:1789760791.485657:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801800415346 time 801826692618 state 0x0 ref 1 00000100:00100000:3.0:1789760791.485671:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493865344:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760791.485719:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28135424, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:3.0:1789760791.485728:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071fd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.485744:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=801s v=(0 0) 00010000:00010000:3.0:1789760791.485750:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.485756:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.485769:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.520817:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974188 is committed 00002000:00010000:1.0:1789760791.521054:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28135424 -> 28139520 00010000:00010000:1.0:1789760791.521070:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.521083:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071fd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.521093:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.521101:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760791.521116:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.521126:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071fd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.521146:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071fd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.521154:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e071fd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28135424->28139519] (req 28135424->28139519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.521341:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493865344:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35668us (35890us total) trans 4294974188 rc 0/0 00000100:00100000:1.0:1789760791.521352:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6899 00000100:00100000:1.0:1789760791.521361:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801800415346 time 801826692618 state 0x2 ref 0 00000100:00100000:0.0:1789760791.525274:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760791.525321:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493865472 00000100:00100000:1.0:1789760791.525331:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493865472 00000100:00100000:1.0:1789760791.525387:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801800415346 time 801826692618 state 0x0 ref 1 00000100:00100000:1.0:1789760791.525412:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801826953034 time 801826692618 state 0x1 ref 1 00000100:00100000:1.0:1789760791.525446:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801826953034 time 801866481756 state 0x0 ref 1 00000100:00100000:1.0:1789760791.525459:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493865472:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760791.525511:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28139520, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:1.0:1789760791.525519:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07204 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.525535:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=801s v=(0 0) 00010000:00010000:1.0:1789760791.525539:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.525545:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07204 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.525562:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07204 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.585726:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974189 is committed 00002000:00010000:1.0:1789760791.586030:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28139520 -> 28143616 00010000:00010000:1.0:1789760791.586045:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07204 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.586058:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07204 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.586067:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07204 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.586074:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760791.586078:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07204 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.586087:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07204 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.586117:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07204 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.586125:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07204 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28139520->28143615] (req 28139520->28143615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.586481:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493865472:12345-192.168.203.60@tcp:4:dd.0 Request processed in 61018us (61335us total) trans 4294974189 rc 0/0 00000100:00100000:1.0:1789760791.586492:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6900 00000100:00100000:1.0:1789760791.586502:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801826953034 time 801866481756 state 0x2 ref 0 00000100:00100000:2.0:1789760791.594338:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760791.594486:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493865600 00000100:00100000:1.0:1789760791.594501:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493865600 00000100:00100000:1.0:1789760791.594570:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801826953034 time 801866481756 state 0x0 ref 1 00000100:00100000:1.0:1789760791.594587:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801866742172 time 801866481756 state 0x1 ref 1 00000100:00100000:1.0:1789760791.594628:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801866742172 time 801935662754 state 0x0 ref 1 00000100:00100000:1.0:1789760791.594641:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493865600:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760791.594691:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28143616, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xd809 00010000:00010000:1.0:1789760791.594701:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0720b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.594717:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=801s v=(0 0) 00010000:00010000:1.0:1789760791.594722:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.594728:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0720b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.594818:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0720b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.653995:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974190 is committed 00002000:00010000:1.0:1789760791.654097:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28143616 -> 28147712 00002000:00010000:1.0:1789760791.654117:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760790 -> 1789760791 00002000:00010000:1.0:1789760791.654120:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760790 -> 1789760791 00010000:00010000:1.0:1789760791.654134:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0720b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.654146:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0720b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.654323:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0720b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.654400:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760791.654405:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0720b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.654415:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0720b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.654436:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0720b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.654443:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0720b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28143616->28147711] (req 28143616->28147711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.654546:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8554e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493865600:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59902us (60218us total) trans 4294974190 rc 0/0 00000100:00100000:1.0:1789760791.654556:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6901 00000100:00100000:1.0:1789760791.654566:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801866742172 time 801935662754 state 0x2 ref 0 00000100:00100000:0.0:1789760791.661074:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760791.663274:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493865728 00000100:00100000:2.0:1789760791.663290:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493865728 00000100:00100000:2.0:1789760791.663354:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801866742172 time 801935662754 state 0x0 ref 1 00000100:00100000:2.0:1789760791.663370:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801935923170 time 801935662754 state 0x1 ref 1 00000100:00100000:2.0:1789760791.663411:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801935923170 time 802004446529 state 0x0 ref 1 00000100:00100000:2.0:1789760791.663425:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493865728:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760791.663475:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28147712, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:2.0:1789760791.663485:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e07212 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.663500:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=802s v=(0 0) 00010000:00010000:2.0:1789760791.663504:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.663510:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e07212 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.663526:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e07212 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.703885:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974191 is committed 00002000:00010000:3.0:1789760791.704040:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28147712 -> 28151808 00010000:00010000:3.0:1789760791.704054:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e07212 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.704072:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e07212 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.704080:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e07212 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.704088:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760791.704093:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e07212 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.704101:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e07212 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.704131:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e07212 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.704147:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e07212 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28147712->28151807] (req 28147712->28151807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.704355:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493865728:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40928us (43286us total) trans 4294974191 rc 0/0 00000100:00100000:3.0:1789760791.704366:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6902 00000100:00100000:3.0:1789760791.704375:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801935923170 time 802004446529 state 0x2 ref 0 00000100:00100000:2.0:1789760791.711419:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760791.711509:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493865856 00000100:00100000:3.0:1789760791.711520:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493865856 00000100:00100000:3.0:1789760791.711572:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 801935923170 time 802004446529 state 0x0 ref 1 00000100:00100000:3.0:1789760791.711588:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802004706945 time 802004446529 state 0x1 ref 1 00000100:00100000:3.0:1789760791.711622:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802004706945 time 802052656821 state 0x0 ref 1 00000100:00100000:3.0:1789760791.711634:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8555880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493865856:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760791.711683:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28151808, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:3.0:1789760791.711693:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07219 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.711708:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=802s v=(0 0) 00010000:00010000:3.0:1789760791.711713:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.711719:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07219 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.711733:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07219 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.754767:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974192 is committed 00002000:00010000:3.0:1789760791.754879:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28151808 -> 28155904 00010000:00010000:3.0:1789760791.754894:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07219 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.755039:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07219 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.755054:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07219 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.755061:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760791.755066:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07219 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.755075:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07219 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.755115:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07219 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.755123:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07219 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28151808->28155903] (req 28151808->28155903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.755206:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8555880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493865856:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43569us (43792us total) trans 4294974192 rc 0/0 00000100:00100000:3.0:1789760791.755225:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6903 00000100:00100000:3.0:1789760791.755393:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802004706945 time 802052656821 state 0x2 ref 0 00000100:00100000:1.0:1789760791.762998:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760791.763852:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493865984 00000100:00100000:2.0:1789760791.763863:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493865984 00000100:00100000:2.0:1789760791.763921:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802004706945 time 802052656821 state 0x0 ref 1 00000100:00100000:2.0:1789760791.763945:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802052917237 time 802052656821 state 0x1 ref 1 00000100:00100000:2.0:1789760791.764009:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802052917237 time 802105044633 state 0x0 ref 1 00000100:00100000:2.0:1789760791.764021:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493865984:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760791.764071:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28155904, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:2.0:1789760791.764079:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e07220 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.764093:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=802s v=(0 0) 00010000:00010000:2.0:1789760791.764098:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.764116:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e07220 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.764131:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e07220 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.801301:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974193 is committed 00002000:00010000:3.0:1789760791.801396:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28155904 -> 28160000 00010000:00010000:3.0:1789760791.801412:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e07220 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.801430:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e07220 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.801440:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e07220 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.801447:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760791.801452:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e07220 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.801463:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e07220 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.801484:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e07220 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.801493:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e07220 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28155904->28159999] (req 28155904->28159999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.801578:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493865984:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37554us (38621us total) trans 4294974193 rc 0/0 00000100:00100000:3.0:1789760791.801588:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6904 00000100:00100000:3.0:1789760791.801598:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802052917237 time 802105044633 state 0x2 ref 0 00000100:00100000:0.0:1789760791.815127:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760791.815229:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493866112 00000100:00100000:1.0:1789760791.815239:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493866112 00000100:00100000:1.0:1789760791.815294:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802052917237 time 802105044633 state 0x0 ref 1 00000100:00100000:1.0:1789760791.815310:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802105305049 time 802105044633 state 0x1 ref 1 00000100:00100000:1.0:1789760791.815341:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802105305049 time 802156376108 state 0x0 ref 1 00000100:00100000:1.0:1789760791.815354:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493866112:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760791.815401:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28160000, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:1.0:1789760791.815410:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07227 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.815426:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=802s v=(0 0) 00010000:00010000:1.0:1789760791.815430:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.815436:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07227 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.815455:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07227 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.871609:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974194 is committed 00002000:00010000:0.0:1789760791.871820:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28160000 -> 28164096 00010000:00010000:0.0:1789760791.871836:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07227 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.871847:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07227 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.871856:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07227 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.871872:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760791.871876:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07227 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.871894:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07227 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.871913:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07227 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.871921:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07227 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28160000->28164095] (req 28160000->28164095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.872045:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493866112:12345-192.168.203.60@tcp:4:dd.0 Request processed in 56689us (56923us total) trans 4294974194 rc 0/0 00000100:00100000:0.0:1789760791.872183:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6905 00000100:00100000:0.0:1789760791.872193:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802105305049 time 802156376108 state 0x2 ref 0 00000100:00100000:2.0:1789760791.887388:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760791.887450:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493866240 00000100:00100000:3.0:1789760791.887460:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493866240 00000100:00100000:3.0:1789760791.887513:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802105305049 time 802156376108 state 0x0 ref 1 00000100:00100000:3.0:1789760791.887528:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802156636524 time 802156376108 state 0x1 ref 1 00000100:00100000:3.0:1789760791.887563:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802156636524 time 802228598172 state 0x0 ref 1 00000100:00100000:3.0:1789760791.887576:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493866240:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760791.887634:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28164096, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:3.0:1789760791.887643:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0722e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.887658:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=802s v=(0 0) 00010000:00010000:3.0:1789760791.887663:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.887669:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0722e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.887682:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0722e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760791.933554:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28164096 -> 28168192 00010000:00010000:3.0:1789760791.933574:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0722e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.933588:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0722e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.933602:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0722e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.933609:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760791.933613:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0722e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.933622:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0722e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.933641:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0722e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.933649:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0722e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28164096->28168191] (req 28164096->28168191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.933732:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493866240:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46153us (46348us total) trans 4294974195 rc 0/0 00000100:00100000:3.0:1789760791.933742:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6906 00000100:00100000:3.0:1789760791.933752:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802156636524 time 802228598172 state 0x2 ref 0 00000001:00080000:2.0:1789760791.933856:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974195 is committed 00000100:00100000:2.0:1789760791.944751:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760791.945873:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493866368 00000100:00100000:3.0:1789760791.945884:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493866368 00000100:00100000:2.0:1789760791.946215:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802156636524 time 802228598172 state 0x0 ref 1 00000100:00100000:2.0:1789760791.946230:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802228858588 time 802228598172 state 0x1 ref 1 00000100:00100000:2.0:1789760791.946295:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802228858588 time 802287330303 state 0x0 ref 1 00000100:00100000:2.0:1789760791.946311:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4529c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493866368:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760791.946358:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28168192, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:2.0:1789760791.946366:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e07235 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.946386:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=802s v=(0 0) 00010000:00010000:2.0:1789760791.946391:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.946396:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e07235 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.946408:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e07235 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.973047:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974196 is committed 00002000:00010000:2.0:1789760791.973261:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28168192 -> 28172288 00010000:00010000:2.0:1789760791.973279:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e07235 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.973291:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e07235 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.973298:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e07235 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.973305:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760791.973309:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e07235 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.973318:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e07235 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.973335:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e07235 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.973343:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e07235 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28168192->28172287] (req 28168192->28172287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.973491:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4529c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493866368:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27179us (28744us total) trans 4294974196 rc 0/0 00000100:00100000:2.0:1789760791.973501:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6907 00000100:00100000:2.0:1789760791.973511:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802228858588 time 802287330303 state 0x2 ref 0 00000100:00100000:0.0:1789760791.980809:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760791.980906:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493866496 00000100:00100000:2.0:1789760791.980917:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493866496 00000100:00100000:2.0:1789760791.981004:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802228858588 time 802287330303 state 0x0 ref 1 00000100:00100000:2.0:1789760791.981092:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802287590719 time 802287330303 state 0x1 ref 1 00000100:00100000:2.0:1789760791.981136:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802287590719 time 802322171209 state 0x0 ref 1 00000100:00100000:2.0:1789760791.981149:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493866496:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760791.981195:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28172288, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:2.0:1789760791.981204:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0723c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.981219:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=802s v=(0 0) 00010000:00010000:2.0:1789760791.981224:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.981230:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0723c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.981248:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0723c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760792.021029:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28172288 -> 28176384 00010000:00010000:2.0:1789760792.021050:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0723c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.021063:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0723c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.021073:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0723c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.021081:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760792.021087:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0723c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.021096:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0723c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.021128:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0723c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.021137:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b171fc37/0x3eb3dcc0b2e0723c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28172288->28176383] (req 28172288->28176383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.021248:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493866496:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40095us (40444us total) trans 4294974197 rc 0/0 00000100:00100000:2.0:1789760792.021259:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6908 00000100:00100000:2.0:1789760792.021269:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802287590719 time 802322171209 state 0x2 ref 0 00000001:00080000:3.0:1789760792.021844:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974197 is committed 00000100:00100000:0.0:1789760792.024921:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760792.027222:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493866624 00000100:00100000:1.0:1789760792.027236:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493866624 00000100:00100000:1.0:1789760792.027294:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802287590719 time 802322171209 state 0x0 ref 1 00000100:00100000:1.0:1789760792.027310:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802322431625 time 802322171209 state 0x1 ref 1 00000100:00100000:1.0:1789760792.027377:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802322431625 time 802368412214 state 0x0 ref 1 00000100:00100000:1.0:1789760792.027390:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493866624:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760792.027444:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28176384, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:1.0:1789760792.027453:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07243 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.027468:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=802s v=(0 0) 00010000:00010000:1.0:1789760792.027472:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.027478:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07243 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.027492:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07243 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.062183:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974198 is committed 00002000:00010000:1.0:1789760792.062322:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28176384 -> 28180480 00010000:00010000:1.0:1789760792.062338:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07243 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.062351:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07243 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.062360:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07243 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.062368:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760792.062373:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07243 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.062384:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07243 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.062598:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07243 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.062607:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07243 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28176384->28180479] (req 28176384->28180479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.062743:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493866624:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35350us (37825us total) trans 4294974198 rc 0/0 00000100:00100000:1.0:1789760792.062753:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6909 00000100:00100000:1.0:1789760792.062763:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802322431625 time 802368412214 state 0x2 ref 0 00000100:00100000:1.0:1789760792.074319:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760792.075454:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493866752 00000100:00100000:3.0:1789760792.075465:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493866752 00000100:00100000:3.0:1789760792.075524:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802322431625 time 802368412214 state 0x0 ref 1 00000100:00100000:3.0:1789760792.075538:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802368672630 time 802368412214 state 0x1 ref 1 00000100:00100000:3.0:1789760792.075691:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802368672630 time 802416724926 state 0x0 ref 1 00000100:00100000:3.0:1789760792.075705:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493866752:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760792.075754:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28180480, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:3.0:1789760792.075764:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0724a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.075780:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=802s v=(0 0) 00010000:00010000:3.0:1789760792.075785:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.075791:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0724a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.075805:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0724a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760792.113901:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28180480 -> 28184576 00010000:00010000:0.0:1789760792.113923:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0724a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.113935:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0724a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.113944:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0724a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.113951:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760792.113957:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0724a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.114000:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0724a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.114020:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0724a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.114029:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0724a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28180480->28184575] (req 28180480->28184575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.114119:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493866752:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38411us (39805us total) trans 4294974199 rc 0/0 00000100:00100000:0.0:1789760792.114129:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6910 00000100:00100000:0.0:1789760792.114138:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802368672630 time 802416724926 state 0x2 ref 0 00000001:00080000:1.0:1789760792.114327:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974199 is committed 00000100:00100000:1.0:1789760792.118350:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760792.118424:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493866880 00000100:00100000:2.0:1789760792.118434:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493866880 00000100:00100000:2.0:1789760792.118489:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802368672630 time 802416724926 state 0x0 ref 1 00000100:00100000:2.0:1789760792.118505:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802416985342 time 802416724926 state 0x1 ref 1 00000100:00100000:2.0:1789760792.118534:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802416985342 time 802459569897 state 0x0 ref 1 00000100:00100000:2.0:1789760792.118547:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493866880:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760792.118603:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28184576, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:2.0:1789760792.118612:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e07251 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.118627:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=802s v=(0 0) 00010000:00010000:2.0:1789760792.118632:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.118638:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e07251 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.118652:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e07251 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.122043:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760792.157581:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974200 is committed 00002000:00010000:2.0:1789760792.157757:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28184576 -> 28188672 00010000:00010000:2.0:1789760792.157773:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e07251 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.157785:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e07251 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.157793:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e07251 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.157801:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760792.157805:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e07251 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.157814:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e07251 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.157833:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e07251 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.157841:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e07251 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28184576->28188671] (req 28184576->28188671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.158001:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493866880:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39416us (39619us total) trans 4294974200 rc 0/0 00000100:00100000:2.0:1789760792.158015:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6911 00000100:00100000:2.0:1789760792.158024:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802416985342 time 802459569897 state 0x2 ref 0 00000100:00100000:2.0:1789760792.162573:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760792.162672:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493867008 00000100:00100000:3.0:1789760792.162681:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493867008 00000100:00100000:3.0:1789760792.162756:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802416985342 time 802459569897 state 0x0 ref 1 00000100:00100000:3.0:1789760792.162775:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802459830313 time 802459569897 state 0x1 ref 1 00000100:00100000:3.0:1789760792.162802:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802459830313 time 802503835063 state 0x0 ref 1 00000100:00100000:3.0:1789760792.162822:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493867008:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760792.162882:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28188672, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:3.0:1789760792.162891:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07258 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.162909:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=802s v=(0 0) 00010000:00010000:3.0:1789760792.162913:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.162923:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07258 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.162937:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07258 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760792.199041:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28188672 -> 28192768 00010000:00010000:0.0:1789760792.199065:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07258 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.199078:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07258 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.199086:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07258 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.199093:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760792.199114:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07258 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.199124:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07258 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.199144:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07258 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.199272:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e07258 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28188672->28192767] (req 28188672->28192767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.199438:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493867008:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36613us (36875us total) trans 4294974201 rc 0/0 00000100:00100000:0.0:1789760792.199452:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6912 00000100:00100000:0.0:1789760792.199463:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802459830313 time 802503835063 state 0x2 ref 0 00000001:00080000:1.0:1789760792.199552:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974201 is committed 00000100:00100000:1.0:1789760792.205924:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760792.206215:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493867136 00000100:00100000:0.0:1789760792.206227:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493867136 00000100:00100000:0.0:1789760792.206284:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802459830313 time 802503835063 state 0x0 ref 1 00000100:00100000:0.0:1789760792.206299:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802504095479 time 802503835063 state 0x1 ref 1 00000100:00100000:0.0:1789760792.206332:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802504095479 time 802547366921 state 0x0 ref 1 00000100:00100000:0.0:1789760792.206346:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493867136:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760792.206395:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28192768, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:0.0:1789760792.206405:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0725f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.206420:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=802s v=(0 0) 00010000:00010000:0.0:1789760792.206425:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.206432:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0725f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.206657:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0725f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760792.242052:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28192768 -> 28196864 00010000:00010000:0.0:1789760792.242071:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0725f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.242085:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0725f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.242094:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0725f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.242114:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760792.242119:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0725f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.242128:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0725f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.242147:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0725f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.242276:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974202 is committed 00010000:00010000:0.0:1789760792.242835:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0725f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28192768->28196863] (req 28192768->28196863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.242938:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493867136:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36589us (37018us total) trans 4294974202 rc 0/0 00000100:00100000:0.0:1789760792.242950:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6913 00000100:00100000:0.0:1789760792.243004:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802504095479 time 802547366921 state 0x2 ref 0 00000100:00100000:1.0:1789760792.247157:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760792.247273:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493867264 00000100:00100000:0.0:1789760792.247283:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493867264 00000100:00100000:0.0:1789760792.247338:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802504095479 time 802547366921 state 0x0 ref 1 00000100:00100000:0.0:1789760792.247353:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802547627337 time 802547366921 state 0x1 ref 1 00000100:00100000:0.0:1789760792.247386:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802547627337 time 802588421597 state 0x0 ref 1 00000100:00100000:0.0:1789760792.247400:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493867264:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760792.247449:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28196864, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:0.0:1789760792.247458:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e07266 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.247473:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=802s v=(0 0) 00010000:00010000:0.0:1789760792.247477:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.247484:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e07266 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.247497:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e07266 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760792.298473:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28196864 -> 28200960 00010000:00010000:0.0:1789760792.298493:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e07266 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.298506:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e07266 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.298519:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e07266 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.298527:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760792.298532:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e07266 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.298541:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e07266 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.298561:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e07266 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.298568:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e07266 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28196864->28200959] (req 28196864->28200959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.298732:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493867264:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51330us (51578us total) trans 4294974203 rc 0/0 00000100:00100000:0.0:1789760792.298742:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6914 00000100:00100000:0.0:1789760792.298751:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802547627337 time 802588421597 state 0x2 ref 0 00000001:00080000:3.0:1789760792.298836:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974203 is committed 00000100:00100000:2.0:1789760792.302724:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760792.302900:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493867392 00000100:00100000:0.0:1789760792.302909:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493867392 00000100:00100000:0.0:1789760792.303035:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802547627337 time 802588421597 state 0x0 ref 1 00000100:00100000:0.0:1789760792.303052:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802588682013 time 802588421597 state 0x1 ref 1 00000100:00100000:0.0:1789760792.303125:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802588682013 time 802644160269 state 0x0 ref 1 00000100:00100000:0.0:1789760792.303138:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493867392:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760792.303247:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28200960, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:0.0:1789760792.303256:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0726d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.303270:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=802s v=(0 0) 00010000:00010000:0.0:1789760792.303275:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.303281:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0726d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.303293:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0726d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.336808:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974204 is committed 00002000:00010000:1.0:1789760792.336922:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28200960 -> 28205056 00010000:00010000:1.0:1789760792.336937:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0726d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.336948:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0726d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.336957:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0726d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.337032:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760792.337036:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0726d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.337044:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0726d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.337062:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0726d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.337073:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0726d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28200960->28205055] (req 28200960->28205055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.337268:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493867392:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34128us (34550us total) trans 4294974204 rc 0/0 00000100:00100000:1.0:1789760792.337278:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6915 00000100:00100000:1.0:1789760792.337287:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802588682013 time 802644160269 state 0x2 ref 0 00000100:00100000:1.0:1789760792.341700:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760792.341789:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493867520 00000100:00100000:3.0:1789760792.341798:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493867520 00000100:00100000:3.0:1789760792.341848:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802588682013 time 802644160269 state 0x0 ref 1 00000100:00100000:3.0:1789760792.341865:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802644420685 time 802644160269 state 0x1 ref 1 00000100:00100000:3.0:1789760792.342047:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802644420685 time 802683081244 state 0x0 ref 1 00000100:00100000:3.0:1789760792.342062:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbedf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493867520:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760792.342118:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28205056, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:3.0:1789760792.342128:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07274 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.342142:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=802s v=(0 0) 00010000:00010000:3.0:1789760792.342146:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.342213:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07274 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.342229:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07274 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.375800:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974205 is committed 00002000:00010000:1.0:1789760792.375943:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28205056 -> 28209152 00010000:00010000:1.0:1789760792.376273:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07274 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.376289:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07274 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.376298:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07274 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.376306:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760792.376310:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07274 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.376319:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07274 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.376343:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07274 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.376351:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07274 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28205056->28209151] (req 28205056->28209151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.376478:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbedf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493867520:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34413us (34780us total) trans 4294974205 rc 0/0 00000100:00100000:1.0:1789760792.376489:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6916 00000100:00100000:1.0:1789760792.376498:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802644420685 time 802683081244 state 0x2 ref 0 00000100:00100000:2.0:1789760792.384412:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760792.384522:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493867648 00000100:00100000:3.0:1789760792.384532:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493867648 00000100:00100000:3.0:1789760792.384592:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802644420685 time 802683081244 state 0x0 ref 1 00000100:00100000:3.0:1789760792.384607:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802683341660 time 802683081244 state 0x1 ref 1 00000100:00100000:3.0:1789760792.384633:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802683341660 time 802725668724 state 0x0 ref 1 00000100:00100000:3.0:1789760792.384644:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493867648:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760792.384690:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28209152, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:3.0:1789760792.384698:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0727b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.384712:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=802s v=(0 0) 00010000:00010000:3.0:1789760792.384717:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.384723:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0727b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.384735:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0727b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.426257:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974206 is committed 00002000:00010000:1.0:1789760792.426302:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28209152 -> 28213248 00010000:00010000:1.0:1789760792.426319:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0727b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.426331:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0727b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.426341:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0727b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.426348:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760792.426353:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0727b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.426368:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0727b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.426390:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0727b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.426399:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e0727b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28209152->28213247] (req 28209152->28213247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.426517:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493867648:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41867us (42108us total) trans 4294974206 rc 0/0 00000100:00100000:1.0:1789760792.426529:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6917 00000100:00100000:1.0:1789760792.426539:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802683341660 time 802725668724 state 0x2 ref 0 00000100:00100000:2.0:1789760792.431201:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760792.431339:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493867776 00000100:00100000:0.0:1789760792.431349:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493867776 00000100:00100000:0.0:1789760792.431411:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802683341660 time 802725668724 state 0x0 ref 1 00000100:00100000:0.0:1789760792.431428:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802725929140 time 802725668724 state 0x1 ref 1 00000100:00100000:0.0:1789760792.431461:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802725929140 time 802772496116 state 0x0 ref 1 00000100:00100000:0.0:1789760792.431474:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493867776:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760792.431533:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28213248, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:0.0:1789760792.431541:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e07282 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.431556:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=802s v=(0 0) 00010000:00010000:0.0:1789760792.431561:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.431567:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e07282 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.431593:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e07282 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.471075:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974207 is committed 00002000:00010000:0.0:1789760792.472056:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28213248 -> 28217344 00010000:00010000:0.0:1789760792.472075:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e07282 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.472088:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e07282 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.472701:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e07282 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.472712:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760792.472717:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e07282 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.472727:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e07282 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.472747:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e07282 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.472756:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e07282 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28213248->28217343] (req 28213248->28217343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.472862:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493867776:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41384us (41665us total) trans 4294974207 rc 0/0 00000100:00100000:0.0:1789760792.472873:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6918 00000100:00100000:0.0:1789760792.472882:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802725929140 time 802772496116 state 0x2 ref 0 00000100:00100000:3.0:1789760792.475761:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760792.475902:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493867904 00000100:00100000:1.0:1789760792.475912:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493867904 00000100:00100000:1.0:1789760792.476012:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802725929140 time 802772496116 state 0x0 ref 1 00000100:00100000:1.0:1789760792.476027:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802772756532 time 802772496116 state 0x1 ref 1 00000100:00100000:1.0:1789760792.476055:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802772756532 time 802817090751 state 0x0 ref 1 00000100:00100000:1.0:1789760792.476068:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3662300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493867904:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760792.476128:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28217344, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:1.0:1789760792.476137:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07289 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.476161:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=802s v=(0 0) 00010000:00010000:1.0:1789760792.476169:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.476175:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07289 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.476188:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07289 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.507057:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974208 is committed 00002000:00010000:1.0:1789760792.507211:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28217344 -> 28221440 00010000:00010000:1.0:1789760792.507226:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07289 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.507238:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07289 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.507248:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07289 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.507255:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760792.507260:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07289 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.507269:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07289 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.507290:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07289 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.507300:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07289 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28217344->28221439] (req 28217344->28221439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.507427:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3662300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493867904:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31356us (31670us total) trans 4294974208 rc 0/0 00000100:00100000:1.0:1789760792.507437:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6919 00000100:00100000:1.0:1789760792.507447:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802772756532 time 802817090751 state 0x2 ref 0 00000100:00100000:2.0:1789760792.516681:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760792.516783:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493868032 00000100:00100000:1.0:1789760792.516793:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493868032 00000100:00100000:1.0:1789760792.516850:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802772756532 time 802817090751 state 0x0 ref 1 00000100:00100000:1.0:1789760792.516866:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802817351167 time 802817090751 state 0x1 ref 1 00000100:00100000:1.0:1789760792.516899:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802817351167 time 802857934508 state 0x0 ref 1 00000100:00100000:1.0:1789760792.516911:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493868032:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760792.517081:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28221440, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:1.0:1789760792.517090:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07290 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.517115:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=802s v=(0 0) 00010000:00010000:1.0:1789760792.517119:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.517125:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07290 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.517144:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07290 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.552771:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974209 is committed 00002000:00010000:1.0:1789760792.553037:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28221440 -> 28225536 00010000:00010000:1.0:1789760792.553052:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07290 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.553066:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07290 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.553075:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07290 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.553083:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760792.553088:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07290 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.553108:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07290 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.553128:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07290 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.553136:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07290 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28221440->28225535] (req 28221440->28225535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.553241:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493868032:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36327us (36566us total) trans 4294974209 rc 0/0 00000100:00100000:1.0:1789760792.553251:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6920 00000100:00100000:1.0:1789760792.553260:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802817351167 time 802857934508 state 0x2 ref 0 00000100:00100000:3.0:1789760792.559987:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760792.560072:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493868160 00000100:00100000:1.0:1789760792.560083:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493868160 00000100:00100000:1.0:1789760792.560147:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802817351167 time 802857934508 state 0x0 ref 1 00000100:00100000:1.0:1789760792.560173:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802858194924 time 802857934508 state 0x1 ref 1 00000100:00100000:1.0:1789760792.560202:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802858194924 time 802901237495 state 0x0 ref 1 00000100:00100000:1.0:1789760792.560214:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc3661500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493868160:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760792.560265:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28225536, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:1.0:1789760792.560275:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07297 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.560299:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=802s v=(0 0) 00010000:00010000:1.0:1789760792.560304:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.560311:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07297 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.560326:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07297 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.592834:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974210 is committed 00002000:00010000:1.0:1789760792.592942:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28225536 -> 28229632 00010000:00010000:1.0:1789760792.592958:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07297 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.592989:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07297 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.592997:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07297 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.593003:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760792.593008:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07297 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.593017:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07297 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.593037:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07297 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.593044:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07297 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28225536->28229631] (req 28225536->28229631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.593146:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc3661500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493868160:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32928us (33187us total) trans 4294974210 rc 0/0 00000100:00100000:1.0:1789760792.593226:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6921 00000100:00100000:1.0:1789760792.593237:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802858194924 time 802901237495 state 0x2 ref 0 00000100:00100000:3.0:1789760792.600854:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760792.601410:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493868288 00000100:00100000:0.0:1789760792.601420:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493868288 00000100:00100000:0.0:1789760792.601488:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802858194924 time 802901237495 state 0x0 ref 1 00000100:00100000:0.0:1789760792.601505:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802901497911 time 802901237495 state 0x1 ref 1 00000100:00100000:0.0:1789760792.601538:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802901497911 time 802942573445 state 0x0 ref 1 00000100:00100000:0.0:1789760792.601549:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4badc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493868288:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760792.601611:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28229632, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xd809 00010000:00010000:0.0:1789760792.601621:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e0729e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.601637:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=802s v=(0 0) 00010000:00010000:0.0:1789760792.601642:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.601648:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e0729e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.601663:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e0729e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.650440:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974211 is committed 00002000:00010000:3.0:1789760792.650662:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28229632 -> 28233728 00002000:00010000:3.0:1789760792.650667:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760791 -> 1789760792 00002000:00010000:3.0:1789760792.650670:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760791 -> 1789760792 00010000:00010000:3.0:1789760792.650682:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e0729e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.650695:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e0729e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.650704:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e0729e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.650711:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760792.650715:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e0729e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.650724:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e0729e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.650748:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e0729e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.650756:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e0729e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28229632->28233727] (req 28229632->28233727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.650883:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4badc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493868288:12345-192.168.203.60@tcp:4:dd.0 Request processed in 49329us (50032us total) trans 4294974211 rc 0/0 00000100:00100000:3.0:1789760792.650893:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6922 00000100:00100000:3.0:1789760792.650901:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802901497911 time 802942573445 state 0x2 ref 0 00000100:00100000:2.0:1789760792.654956:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760792.655118:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493868416 00000100:00100000:1.0:1789760792.655128:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493868416 00000100:00100000:1.0:1789760792.655193:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802901497911 time 802942573445 state 0x0 ref 1 00000100:00100000:1.0:1789760792.655208:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802942833861 time 802942573445 state 0x1 ref 1 00000100:00100000:1.0:1789760792.655234:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802942833861 time 802996269714 state 0x0 ref 1 00000100:00100000:1.0:1789760792.655246:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb14050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493868416:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760792.655295:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28233728, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:1.0:1789760792.655304:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e072a5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.655322:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=802s v=(0 0) 00010000:00010000:1.0:1789760792.655326:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.655332:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e072a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.655345:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e072a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.686122:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974212 is committed 00002000:00010000:3.0:1789760792.686313:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28233728 -> 28237824 00010000:00010000:3.0:1789760792.686328:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e072a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.686342:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e072a5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.686352:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e072a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.686359:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760792.686363:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e072a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.686377:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e072a5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.686399:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e072a5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.686407:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e072a5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28233728->28237823] (req 28233728->28237823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.686595:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb14050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493868416:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31346us (31644us total) trans 4294974212 rc 0/0 00000100:00100000:3.0:1789760792.686607:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6923 00000100:00100000:3.0:1789760792.686616:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802942833861 time 802996269714 state 0x2 ref 0 00000100:00100000:0.0:1789760792.690866:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760792.691082:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493868544 00000100:00100000:1.0:1789760792.691091:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493868544 00000100:00100000:1.0:1789760792.692424:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802942833861 time 802996269714 state 0x0 ref 1 00000100:00100000:1.0:1789760792.692445:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802996530130 time 802996269714 state 0x1 ref 1 00000100:00100000:1.0:1789760792.692852:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802996530130 time 803033885731 state 0x0 ref 1 00000100:00100000:1.0:1789760792.692874:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493868544:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760792.692924:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28237824, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:1.0:1789760792.692933:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e072ac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.692949:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=803s v=(0 0) 00010000:00010000:1.0:1789760792.692954:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.692977:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e072ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.692992:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e072ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.728183:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974213 is committed 00002000:00010000:3.0:1789760792.728296:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28237824 -> 28241920 00010000:00010000:3.0:1789760792.728314:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e072ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.728585:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e072ac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.728594:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e072ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.728601:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760792.728606:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e072ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.728616:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e072ac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.728637:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e072ac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.728645:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e072ac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28237824->28241919] (req 28237824->28241919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.728796:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493868544:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35927us (37935us total) trans 4294974213 rc 0/0 00000100:00100000:3.0:1789760792.728807:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6924 00000100:00100000:3.0:1789760792.728817:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802996530130 time 803033885731 state 0x2 ref 0 00000100:00100000:1.0:1789760792.743841:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760792.743931:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493868672 00000100:00100000:3.0:1789760792.743942:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493868672 00000100:00100000:3.0:1789760792.744037:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 802996530130 time 803033885731 state 0x0 ref 1 00000100:00100000:3.0:1789760792.744054:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803034146147 time 803033885731 state 0x1 ref 1 00000100:00100000:3.0:1789760792.744086:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803034146147 time 803085121246 state 0x0 ref 1 00000100:00100000:3.0:1789760792.744110:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493868672:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760792.744170:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28241920, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:3.0:1789760792.744179:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072b3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.744194:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=803s v=(0 0) 00010000:00010000:3.0:1789760792.744199:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.744206:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.744221:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760792.778799:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28241920 -> 28246016 00010000:00010000:3.0:1789760792.778818:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.778832:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072b3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.778842:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.778849:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760792.778854:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.778864:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072b3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.778889:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072b3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.778898:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072b3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28241920->28246015] (req 28241920->28246015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.779017:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493868672:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34915us (35181us total) trans 4294974214 rc 0/0 00000100:00100000:3.0:1789760792.779029:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6925 00000100:00100000:3.0:1789760792.779041:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803034146147 time 803085121246 state 0x2 ref 0 00000001:00080000:1.0:1789760792.779536:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974214 is committed 00000100:00100000:1.0:1789760792.789761:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760792.789920:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493868800 00000100:00100000:3.0:1789760792.789930:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493868800 00000100:00100000:3.0:1789760792.790010:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803034146147 time 803085121246 state 0x0 ref 1 00000100:00100000:3.0:1789760792.790024:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803085381662 time 803085121246 state 0x1 ref 1 00000100:00100000:3.0:1789760792.790050:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803085381662 time 803131086941 state 0x0 ref 1 00000100:00100000:3.0:1789760792.790062:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493868800:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760792.790122:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28246016, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:3.0:1789760792.790130:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072ba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.790145:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=803s v=(0 0) 00010000:00010000:3.0:1789760792.790277:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.790285:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.790299:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760792.833837:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28246016 -> 28250112 00010000:00010000:3.0:1789760792.833862:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.833876:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072ba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.833885:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.833893:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760792.833899:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.833907:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072ba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.833926:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072ba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.833933:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072ba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28246016->28250111] (req 28246016->28250111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.834073:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493868800:12345-192.168.203.60@tcp:4:dd.0 Request processed in 44007us (44315us total) trans 4294974215 rc 0/0 00000100:00100000:3.0:1789760792.834084:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6926 00000100:00100000:3.0:1789760792.834107:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803085381662 time 803131086941 state 0x2 ref 0 00000001:00080000:1.0:1789760792.834446:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974215 is committed 00000100:00100000:3.0:1789760792.838416:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760792.838669:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493868928 00000100:00100000:1.0:1789760792.838681:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493868928 00000100:00100000:1.0:1789760792.838750:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803085381662 time 803131086941 state 0x0 ref 1 00000100:00100000:1.0:1789760792.838771:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803131347357 time 803131086941 state 0x1 ref 1 00000100:00100000:1.0:1789760792.838803:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803131347357 time 803179838711 state 0x0 ref 1 00000100:00100000:1.0:1789760792.838817:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4badc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493868928:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760792.838875:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28250112, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:1.0:1789760792.838885:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e072c1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.838905:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=803s v=(0 0) 00010000:00010000:1.0:1789760792.838909:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.838916:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e072c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.838930:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e072c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.870238:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974216 is committed 00002000:00010000:1.0:1789760792.870349:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28250112 -> 28254208 00010000:00010000:1.0:1789760792.870365:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e072c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.870377:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e072c1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.870386:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e072c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.870399:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760792.870404:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e072c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.870413:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e072c1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.870432:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e072c1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.870439:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e072c1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28250112->28254207] (req 28250112->28254207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.870533:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4badc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493868928:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31715us (32131us total) trans 4294974216 rc 0/0 00000100:00100000:1.0:1789760792.870543:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6927 00000100:00100000:1.0:1789760792.870553:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803131347357 time 803179838711 state 0x2 ref 0 00000100:00100000:3.0:1789760792.874001:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760792.874084:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493869056 00000100:00100000:1.0:1789760792.874104:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493869056 00000100:00100000:1.0:1789760792.874160:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803131347357 time 803179838711 state 0x0 ref 1 00000100:00100000:1.0:1789760792.874173:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803180099127 time 803179838711 state 0x1 ref 1 00000100:00100000:1.0:1789760792.874200:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803180099127 time 803215236088 state 0x0 ref 1 00000100:00100000:1.0:1789760792.874216:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493869056:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760792.874265:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28254208, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:1.0:1789760792.874273:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e072c8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.874289:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=803s v=(0 0) 00010000:00010000:1.0:1789760792.874294:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.874300:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e072c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.874313:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e072c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760792.910439:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28254208 -> 28258304 00010000:00010000:1.0:1789760792.910462:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e072c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.910475:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e072c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.910484:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e072c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.910492:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760792.910496:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e072c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.910507:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e072c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.910528:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e072c8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.910536:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e072c8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28254208->28258303] (req 28254208->28258303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.910748:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974217 is committed 00000100:00100000:1.0:1789760792.910801:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493869056:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36582us (36845us total) trans 4294974217 rc 0/0 00000100:00100000:1.0:1789760792.910812:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6928 00000100:00100000:1.0:1789760792.910826:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803180099127 time 803215236088 state 0x2 ref 0 00000100:00100000:3.0:1789760792.918538:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760792.918732:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493869184 00000100:00100000:2.0:1789760792.918742:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493869184 00000100:00100000:2.0:1789760792.918798:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803180099127 time 803215236088 state 0x0 ref 1 00000100:00100000:2.0:1789760792.918814:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803215496504 time 803215236088 state 0x1 ref 1 00000100:00100000:2.0:1789760792.918845:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803215496504 time 803259880678 state 0x0 ref 1 00000100:00100000:2.0:1789760792.918857:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493869184:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760792.918905:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28258304, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:2.0:1789760792.918914:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e072cf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.918935:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=803s v=(0 0) 00010000:00010000:2.0:1789760792.918940:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.919832:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e072cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.919854:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e072cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.956489:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974218 is committed 00002000:00010000:3.0:1789760792.956750:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28258304 -> 28262400 00010000:00010000:3.0:1789760792.956766:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e072cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.956779:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e072cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.956787:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e072cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.956795:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760792.956799:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e072cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.956806:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e072cf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.956830:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e072cf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.956838:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e072cf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28258304->28262399] (req 28258304->28262399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.957042:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493869184:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38172us (38499us total) trans 4294974218 rc 0/0 00000100:00100000:3.0:1789760792.957053:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6929 00000100:00100000:3.0:1789760792.957063:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803215496504 time 803259880678 state 0x2 ref 0 00000100:00100000:3.0:1789760792.969075:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760792.969164:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493869312 00000100:00100000:0.0:1789760792.969173:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493869312 00000100:00100000:0.0:1789760792.969226:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803215496504 time 803259880678 state 0x0 ref 1 00000100:00100000:0.0:1789760792.969241:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803260141094 time 803259880678 state 0x1 ref 1 00000100:00100000:0.0:1789760792.969321:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803260141094 time 803310356598 state 0x0 ref 1 00000100:00100000:0.0:1789760792.969334:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493869312:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760792.969381:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28262400, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:0.0:1789760792.969389:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e072d6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.969404:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=803s v=(0 0) 00010000:00010000:0.0:1789760792.969409:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.969414:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e072d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.969427:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e072d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.000504:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974219 is committed 00002000:00010000:3.0:1789760793.000866:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28262400 -> 28266496 00010000:00010000:3.0:1789760793.000951:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e072d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.000984:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e072d6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.000998:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e072d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.001005:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760793.001010:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e072d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.001025:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e072d6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.001042:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e072d6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.001050:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e072d6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28262400->28266495] (req 28262400->28266495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.001250:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493869312:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31912us (32182us total) trans 4294974219 rc 0/0 00000100:00100000:3.0:1789760793.001644:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6930 00000100:00100000:3.0:1789760793.001654:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803260141094 time 803310356598 state 0x2 ref 0 00000100:00100000:1.0:1789760793.009824:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760793.009939:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493869440 00000100:00100000:3.0:1789760793.009949:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493869440 00000100:00100000:3.0:1789760793.010048:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803260141094 time 803310356598 state 0x0 ref 1 00000100:00100000:3.0:1789760793.010063:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803310617014 time 803310356598 state 0x1 ref 1 00000100:00100000:3.0:1789760793.010085:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803310617014 time 803351121275 state 0x0 ref 1 00000100:00100000:3.0:1789760793.010108:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbeea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493869440:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760793.010170:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28266496, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:3.0:1789760793.010180:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072dd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.010195:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=803s v=(0 0) 00010000:00010000:3.0:1789760793.010199:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.010205:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.010217:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760793.010774:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760793.056268:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974220 is committed 00002000:00010000:3.0:1789760793.056471:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28266496 -> 28270592 00010000:00010000:3.0:1789760793.056487:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.056499:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072dd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.056517:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.056525:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760793.056530:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.056540:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072dd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.056559:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072dd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.056567:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072dd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28266496->28270591] (req 28266496->28270591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.056657:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbeea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493869440:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46557us (46838us total) trans 4294974220 rc 0/0 00000100:00100000:3.0:1789760793.056669:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6931 00000100:00100000:3.0:1789760793.056679:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803310617014 time 803351121275 state 0x2 ref 0 00000100:00100000:1.0:1789760793.061521:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760793.061745:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493869568 00000100:00100000:3.0:1789760793.061756:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493869568 00000100:00100000:3.0:1789760793.061810:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803310617014 time 803351121275 state 0x0 ref 1 00000100:00100000:3.0:1789760793.061825:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803351381691 time 803351121275 state 0x1 ref 1 00000100:00100000:3.0:1789760793.061859:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803351381691 time 803402893980 state 0x0 ref 1 00000100:00100000:3.0:1789760793.061872:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493869568:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760793.061930:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28270592, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:3.0:1789760793.061938:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e072e4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.061954:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=803s v=(0 0) 00010000:00010000:3.0:1789760793.061999:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.062014:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e072e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.062034:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e072e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760793.095254:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28270592 -> 28274688 00010000:00010000:1.0:1789760793.095274:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e072e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.095343:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e072e4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.095358:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e072e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.095366:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760793.095371:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e072e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.095380:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e072e4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.095398:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e072e4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.095405:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e072e4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28270592->28274687] (req 28270592->28274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.095496:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493869568:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33621us (33979us total) trans 4294974221 rc 0/0 00000100:00100000:1.0:1789760793.095505:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6932 00000100:00100000:1.0:1789760793.095514:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803351381691 time 803402893980 state 0x2 ref 0 00000001:00080000:1.0:1789760793.095550:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974221 is committed 00000100:00100000:0.0:1789760793.104714:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760793.104803:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493869696 00000100:00100000:3.0:1789760793.104812:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493869696 00000100:00100000:3.0:1789760793.104864:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803351381691 time 803402893980 state 0x0 ref 1 00000100:00100000:3.0:1789760793.104879:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803403154396 time 803402893980 state 0x1 ref 1 00000100:00100000:3.0:1789760793.104908:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803403154396 time 803445943235 state 0x0 ref 1 00000100:00100000:3.0:1789760793.104920:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493869696:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760793.105008:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28274688, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:3.0:1789760793.105018:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072eb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.105038:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=803s v=(0 0) 00010000:00010000:3.0:1789760793.105043:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.105049:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.105064:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.136589:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974222 is committed 00002000:00010000:0.0:1789760793.136681:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28274688 -> 28278784 00010000:00010000:0.0:1789760793.136695:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.136708:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072eb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.136716:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.136723:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760793.136727:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.136735:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072eb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.136754:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072eb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.136762:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e072eb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28274688->28278783] (req 28274688->28278783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.136864:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493869696:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31942us (32156us total) trans 4294974222 rc 0/0 00000100:00100000:0.0:1789760793.136874:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6933 00000100:00100000:0.0:1789760793.136884:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803403154396 time 803445943235 state 0x2 ref 0 00000100:00100000:0.0:1789760793.145540:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00010000:00010000:1.0:1789760793.147153:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760793.149431:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493869824 00000100:00100000:3.0:1789760793.149447:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493869824 00000100:00100000:3.0:1789760793.149513:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803403154396 time 803445943235 state 0x0 ref 1 00000100:00100000:3.0:1789760793.149529:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803446203651 time 803445943235 state 0x1 ref 1 00000100:00100000:3.0:1789760793.149567:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803446203651 time 803490602568 state 0x0 ref 1 00000100:00100000:3.0:1789760793.149581:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493869824:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760793.149637:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28278784, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:3.0:1789760793.149647:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e072f2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.149663:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=803s v=(0 0) 00010000:00010000:3.0:1789760793.149667:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.149673:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e072f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.149686:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e072f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.182577:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974223 is committed 00002000:00010000:2.0:1789760793.182865:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28278784 -> 28282880 00010000:00010000:2.0:1789760793.182880:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e072f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.182892:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e072f2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.182900:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e072f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.182907:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760793.182912:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e072f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.182921:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e072f2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.182940:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e072f2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.182947:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e072f2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28278784->28282879] (req 28278784->28282879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.183062:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493869824:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33479us (37527us total) trans 4294974223 rc 0/0 00000100:00100000:2.0:1789760793.183077:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6934 00000100:00100000:2.0:1789760793.183087:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803446203651 time 803490602568 state 0x2 ref 0 00000100:00100000:1.0:1789760793.187878:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760793.187997:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493869952 00000100:00100000:0.0:1789760793.188007:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493869952 00000100:00100000:0.0:1789760793.188064:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803446203651 time 803490602568 state 0x0 ref 1 00000100:00100000:0.0:1789760793.188080:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803490862984 time 803490602568 state 0x1 ref 1 00000100:00100000:0.0:1789760793.188123:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803490862984 time 803529157851 state 0x0 ref 1 00000100:00100000:0.0:1789760793.188136:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493869952:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760793.188183:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28282880, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:0.0:1789760793.188192:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e072f9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.188207:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=803s v=(0 0) 00010000:00010000:0.0:1789760793.188212:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.188219:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e072f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.188233:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e072f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.225417:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974224 is committed 00002000:00010000:0.0:1789760793.225549:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28282880 -> 28286976 00010000:00010000:0.0:1789760793.225564:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e072f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.225576:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e072f9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.225585:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e072f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.225592:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760793.225597:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e072f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.225607:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e072f9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.225628:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e072f9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.225636:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e072f9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28282880->28286975] (req 28282880->28286975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.225730:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493869952:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37592us (37857us total) trans 4294974224 rc 0/0 00000100:00100000:0.0:1789760793.225745:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6935 00000100:00100000:0.0:1789760793.225756:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803490862984 time 803529157851 state 0x2 ref 0 00000100:00100000:0.0:1789760793.230876:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760793.231314:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493870080 00000100:00100000:1.0:1789760793.231324:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493870080 00000100:00100000:1.0:1789760793.231383:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803490862984 time 803529157851 state 0x0 ref 1 00000100:00100000:1.0:1789760793.231400:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803529418267 time 803529157851 state 0x1 ref 1 00000100:00100000:1.0:1789760793.231435:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803529418267 time 803572470900 state 0x0 ref 1 00000100:00100000:1.0:1789760793.231450:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620f39f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493870080:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760793.231507:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28286976, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:1.0:1789760793.231516:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07300 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.231532:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=803s v=(0 0) 00010000:00010000:1.0:1789760793.231537:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.231543:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07300 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.231557:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07300 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.265872:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974225 is committed 00002000:00010000:0.0:1789760793.266342:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28286976 -> 28291072 00010000:00010000:0.0:1789760793.266358:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07300 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.266372:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07300 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.266381:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07300 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.266389:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760793.266393:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07300 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.266403:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07300 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.266422:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07300 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.266430:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07300 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28286976->28291071] (req 28286976->28291071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.266630:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620f39f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493870080:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35179us (35760us total) trans 4294974225 rc 0/0 00000100:00100000:0.0:1789760793.266642:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6936 00000100:00100000:0.0:1789760793.266653:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803529418267 time 803572470900 state 0x2 ref 0 00000100:00100000:3.0:1789760793.269190:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760793.269771:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493870208 00000100:00100000:0.0:1789760793.269782:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493870208 00000100:00100000:0.0:1789760793.269836:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803529418267 time 803572470900 state 0x0 ref 1 00000100:00100000:0.0:1789760793.269851:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803572731316 time 803572470900 state 0x1 ref 1 00000100:00100000:0.0:1789760793.269936:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803572731316 time 803610970508 state 0x0 ref 1 00000100:00100000:0.0:1789760793.269949:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493870208:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760793.270041:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28291072, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:0.0:1789760793.270051:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07307 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.270064:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=803s v=(0 0) 00010000:00010000:0.0:1789760793.270069:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.270074:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07307 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.270089:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07307 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.309366:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974226 is committed 00002000:00010000:3.0:1789760793.309549:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28291072 -> 28295168 00010000:00010000:3.0:1789760793.309565:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07307 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.309582:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07307 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.309591:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07307 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.309598:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760793.309603:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07307 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.309612:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07307 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.309632:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07307 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.309640:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07307 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28291072->28295167] (req 28291072->28295167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.309742:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493870208:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39790us (40557us total) trans 4294974226 rc 0/0 00000100:00100000:3.0:1789760793.309754:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6937 00000100:00100000:3.0:1789760793.309763:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803572731316 time 803610970508 state 0x2 ref 0 00000100:00100000:1.0:1789760793.314926:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760793.315589:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493870336 00000100:00100000:3.0:1789760793.315599:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493870336 00000100:00100000:3.0:1789760793.315655:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803572731316 time 803610970508 state 0x0 ref 1 00000100:00100000:3.0:1789760793.315670:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803611230924 time 803610970508 state 0x1 ref 1 00000100:00100000:3.0:1789760793.315701:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803611230924 time 803656736913 state 0x0 ref 1 00000100:00100000:3.0:1789760793.315714:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493870336:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760793.315767:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28295168, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:3.0:1789760793.315775:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0730e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.315790:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=803s v=(0 0) 00010000:00010000:3.0:1789760793.315795:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.315801:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0730e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.315815:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0730e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.361635:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974227 is committed 00002000:00010000:1.0:1789760793.361862:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28295168 -> 28299264 00010000:00010000:1.0:1789760793.361875:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0730e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.361892:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0730e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.361900:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0730e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.361906:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760793.361910:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0730e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.361918:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0730e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.361935:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0730e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.361943:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0730e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28295168->28299263] (req 28295168->28299263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.362066:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493870336:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46349us (47144us total) trans 4294974227 rc 0/0 00000100:00100000:1.0:1789760793.362076:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6938 00000100:00100000:1.0:1789760793.362086:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803611230924 time 803656736913 state 0x2 ref 0 00000100:00100000:0.0:1789760793.366396:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760793.366431:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493870464 00000100:00100000:2.0:1789760793.366442:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493870464 00000100:00100000:2.0:1789760793.366499:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803611230924 time 803656736913 state 0x0 ref 1 00000100:00100000:2.0:1789760793.366514:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803656997329 time 803656736913 state 0x1 ref 1 00000100:00100000:2.0:1789760793.366546:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803656997329 time 803707581736 state 0x0 ref 1 00000100:00100000:2.0:1789760793.366559:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493870464:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760793.366605:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28299264, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:2.0:1789760793.366613:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e07315 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.366629:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=803s v=(0 0) 00010000:00010000:2.0:1789760793.366634:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.366640:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e07315 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.366669:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e07315 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.394263:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974228 is committed 00002000:00010000:1.0:1789760793.394352:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28299264 -> 28303360 00010000:00010000:1.0:1789760793.394365:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e07315 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.394374:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e07315 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.394380:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e07315 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.394385:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193cc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760793.394388:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e07315 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.394394:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e07315 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.394409:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e07315 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.394416:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000023576c87/0x3eb3dcc0b2e07315 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28299264->28303359] (req 28299264->28303359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.394501:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493870464:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27940us (28116us total) trans 4294974228 rc 0/0 00000100:00100000:1.0:1789760793.394511:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6939 00000100:00100000:1.0:1789760793.394520:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803656997329 time 803707581736 state 0x2 ref 0 00000100:00100000:3.0:1789760793.397807:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760793.398356:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493870592 00000100:00100000:0.0:1789760793.398366:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493870592 00000100:00100000:0.0:1789760793.398420:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803656997329 time 803707581736 state 0x0 ref 1 00000100:00100000:0.0:1789760793.398434:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803707842152 time 803707581736 state 0x1 ref 1 00000100:00100000:0.0:1789760793.398503:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803707842152 time 803739537452 state 0x0 ref 1 00000100:00100000:0.0:1789760793.398517:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493870592:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760793.398565:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28303360, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:0.0:1789760793.398585:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0731c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.398602:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=803s v=(0 0) 00010000:00010000:0.0:1789760793.398607:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.398613:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0731c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.398627:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0731c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760793.437859:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28303360 -> 28307456 00010000:00010000:1.0:1789760793.437879:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0731c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.437891:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0731c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.437901:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0731c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.437908:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760793.437913:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0731c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.437922:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0731c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.437942:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0731c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.437950:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0731c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28303360->28307455] (req 28303360->28307455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.438066:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493870592:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39546us (40262us total) trans 4294974229 rc 0/0 00000100:00100000:1.0:1789760793.438077:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6940 00000100:00100000:1.0:1789760793.438086:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803707842152 time 803739537452 state 0x2 ref 0 00000001:00080000:2.0:1789760793.439255:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974229 is committed 00000100:00100000:3.0:1789760793.440784:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760793.440881:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493870720 00000100:00100000:0.0:1789760793.440890:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493870720 00000100:00100000:0.0:1789760793.440942:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803707842152 time 803739537452 state 0x0 ref 1 00000100:00100000:0.0:1789760793.440956:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803739797868 time 803739537452 state 0x1 ref 1 00000100:00100000:0.0:1789760793.441009:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803739797868 time 803782045212 state 0x0 ref 1 00000100:00100000:0.0:1789760793.441022:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5834000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493870720:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760793.441067:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28307456, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:0.0:1789760793.441076:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e07323 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.441099:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=803s v=(0 0) 00010000:00010000:0.0:1789760793.441104:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.441110:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e07323 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.441124:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e07323 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.481798:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974230 is committed 00002000:00010000:3.0:1789760793.481877:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28307456 -> 28311552 00010000:00010000:3.0:1789760793.481893:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e07323 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.481906:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e07323 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.481917:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e07323 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.481924:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760793.481928:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e07323 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.481938:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e07323 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.481956:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e07323 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.482243:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e07323 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28307456->28311551] (req 28307456->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.482428:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5834000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493870720:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41404us (41645us total) trans 4294974230 rc 0/0 00000100:00100000:3.0:1789760793.482447:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6941 00000100:00100000:3.0:1789760793.482456:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803739797868 time 803782045212 state 0x2 ref 0 00000100:00100000:0.0:1789760793.489255:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760793.489391:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493870848 00000100:00100000:1.0:1789760793.489401:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493870848 00000100:00100000:1.0:1789760793.489454:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803739797868 time 803782045212 state 0x0 ref 1 00000100:00100000:1.0:1789760793.489468:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803782305628 time 803782045212 state 0x1 ref 1 00000100:00100000:1.0:1789760793.489498:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803782305628 time 803830534237 state 0x0 ref 1 00000100:00100000:1.0:1789760793.489510:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493870848:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760793.489556:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28311552, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xd809 00010000:00010000:1.0:1789760793.489564:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0732a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.489578:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=803s v=(0 0) 00010000:00010000:1.0:1789760793.489582:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.489588:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0732a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.489601:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0732a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.526599:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974231 is committed 00002000:00010000:1.0:1789760793.526737:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28311552 -> 28315648 00002000:00010000:1.0:1789760793.526742:0:6692:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb blocks from disk: 55305 -> 57353 00010000:00010000:1.0:1789760793.526755:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0732a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.526772:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0732a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.526781:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0732a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.526787:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760793.526792:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0732a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.526802:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0732a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.526820:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0732a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.526828:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e0732a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28311552->28315647] (req 28311552->28315647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.526924:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493870848:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37409us (37672us total) trans 4294974231 rc 0/0 00000100:00100000:1.0:1789760793.526935:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6942 00000100:00100000:1.0:1789760793.526946:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803782305628 time 803830534237 state 0x2 ref 0 00000100:00100000:1.0:1789760793.540639:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760793.540769:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493870976 00000100:00100000:0.0:1789760793.540779:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493870976 00000100:00100000:0.0:1789760793.540834:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803782305628 time 803830534237 state 0x0 ref 1 00000100:00100000:0.0:1789760793.540849:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803830794653 time 803830534237 state 0x1 ref 1 00000100:00100000:0.0:1789760793.540885:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803830794653 time 803881920069 state 0x0 ref 1 00000100:00100000:0.0:1789760793.540896:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbbedf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493870976:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760793.540957:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28315648, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xe009 00010000:00010000:0.0:1789760793.540985:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e07331 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.541001:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=803s v=(0 0) 00010000:00010000:0.0:1789760793.541005:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.541011:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e07331 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.541023:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e07331 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.582364:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974232 is committed 00002000:00010000:2.0:1789760793.582473:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28315648 -> 28319744 00010000:00010000:2.0:1789760793.582489:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e07331 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.582501:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e07331 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.582511:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e07331 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.582518:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760793.582524:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e07331 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.582534:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e07331 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.582552:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e07331 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.582560:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e07331 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28315648->28319743] (req 28315648->28319743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.582647:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbbedf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493870976:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41747us (42011us total) trans 4294974232 rc 0/0 00000100:00100000:2.0:1789760793.582658:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6943 00000100:00100000:2.0:1789760793.582667:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803830794653 time 803881920069 state 0x2 ref 0 00000100:00100000:3.0:1789760793.585407:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760793.585491:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493871104 00000100:00100000:0.0:1789760793.585506:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493871104 00000100:00100000:0.0:1789760793.585558:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803830794653 time 803881920069 state 0x0 ref 1 00000100:00100000:0.0:1789760793.585573:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803882180485 time 803881920069 state 0x1 ref 1 00000100:00100000:0.0:1789760793.585606:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803882180485 time 803926641639 state 0x0 ref 1 00000100:00100000:0.0:1789760793.585618:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493871104:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760793.585669:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28319744, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xe009 00010000:00010000:0.0:1789760793.585677:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e07338 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.585692:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=803s v=(0 0) 00010000:00010000:0.0:1789760793.585697:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.585703:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e07338 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.585718:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e07338 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760793.616520:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28319744 -> 28323840 00002000:00010000:0.0:1789760793.616528:0:6693:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760792 -> 1789760793 00002000:00010000:0.0:1789760793.616531:0:6693:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760792 -> 1789760793 00010000:00010000:0.0:1789760793.616544:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e07338 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.616556:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e07338 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.616565:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e07338 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.616572:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760793.616577:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e07338 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.616586:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e07338 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.616606:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e07338 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.616614:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e07338 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28319744->28323839] (req 28319744->28323839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.616709:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493871104:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31087us (31303us total) trans 4294974233 rc 0/0 00000100:00100000:0.0:1789760793.616719:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6944 00000100:00100000:0.0:1789760793.616729:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803882180485 time 803926641639 state 0x2 ref 0 00000001:00080000:1.0:1789760793.617570:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974233 is committed 00000100:00100000:0.0:1789760793.622414:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760793.622509:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493871232 00000100:00100000:3.0:1789760793.622519:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493871232 00000100:00100000:3.0:1789760793.622577:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803882180485 time 803926641639 state 0x0 ref 1 00000100:00100000:3.0:1789760793.622595:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803926902055 time 803926641639 state 0x1 ref 1 00000100:00100000:3.0:1789760793.622628:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803926902055 time 803963663170 state 0x0 ref 1 00000100:00100000:3.0:1789760793.622771:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493871232:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760793.622818:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28323840, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe009 00010000:00010000:3.0:1789760793.622825:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0733f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.622838:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=803s v=(0 0) 00010000:00010000:3.0:1789760793.622842:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.622846:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0733f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.622856:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0733f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760793.658276:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28323840 -> 28327936 00010000:00010000:3.0:1789760793.658296:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0733f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.658309:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0733f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.658317:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0733f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.658330:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760793.658334:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0733f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.658343:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0733f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.658361:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0733f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.658368:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0733f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28323840->28327935] (req 28323840->28327935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.658471:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493871232:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35827us (36071us total) trans 4294974234 rc 0/0 00000100:00100000:3.0:1789760793.658609:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6945 00000100:00100000:3.0:1789760793.658619:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803926902055 time 803963663170 state 0x2 ref 0 00000001:00080000:0.0:1789760793.659055:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974234 is committed 00000100:00100000:2.0:1789760793.662286:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760793.662387:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493871360 00000100:00100000:1.0:1789760793.662399:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493871360 00000100:00100000:1.0:1789760793.662525:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803926902055 time 803963663170 state 0x0 ref 1 00000100:00100000:1.0:1789760793.662541:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803963923586 time 803963663170 state 0x1 ref 1 00000100:00100000:1.0:1789760793.662582:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803963923586 time 804003617309 state 0x0 ref 1 00000100:00100000:1.0:1789760793.662595:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493871360:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760793.662642:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28327936, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe009 00010000:00010000:1.0:1789760793.662651:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07346 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.662667:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=804s v=(0 0) 00010000:00010000:1.0:1789760793.662671:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.662678:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07346 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.662693:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07346 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.705051:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760793.705085:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760793.705133:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760793.705165:0:10519:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493871744 00000100:00100000:0.0:1789760793.705174:0:10519:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493871744 00000100:00100000:0.0:1789760793.705239:0:10519:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493871616 00000100:00100000:0.0:1789760793.705243:0:10519:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493871616 00000100:00100000:0.0:1789760793.705268:0:10519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 532 00000100:00100000:0.0:1789760793.705277:0:10519:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe84e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2367:x1876699493871744:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:0.0:1789760793.705349:0:10519:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe84e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:78fddf19-6995-4f97-8b3a-c5dc3cb1d098+5:2367:x1876699493871744:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 73us (263us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760793.705358:0:10519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 532 00000100:00100000:0.0:1789760793.705374:0:10519:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 533 00000100:00100000:0.0:1789760793.705381:0:10519:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2367:x1876699493871616:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:0.0:1789760793.705405:0:10519:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:9b240726-48d9-467b-83a8-5b29f6d3cb3b+8:2367:x1876699493871616:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 24us (271us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760793.705410:0:10519:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 533 00000100:00080000:0.0:1789760793.705429:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760793.705761:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493871488 00000100:00100000:3.0:1789760793.705767:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493871488 00000100:00100000:3.0:1789760793.705854:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.60@tcp, seq: 419 00000100:00100000:3.0:1789760793.705862:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d07850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2367:x1876699493871488:12345-192.168.203.60@tcp:400:kworker.0 00000100:00100000:3.0:1789760793.705900:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d07850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:96f72786-7ebd-4df8-8048-c06aee14e87f+13:2367:x1876699493871488:12345-192.168.203.60@tcp:400:kworker.0 Request processed in 39us (859us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760793.705906:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.60@tcp, seq: 419 00002000:00010000:0.0:1789760793.735698:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28327936 -> 28332032 00010000:00010000:0.0:1789760793.735719:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07346 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.735732:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07346 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.735741:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07346 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.735748:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760793.735753:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07346 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.735763:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07346 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.735783:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07346 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.735792:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e07346 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28327936->28332031] (req 28327936->28332031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.735891:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493871360:12345-192.168.203.60@tcp:4:dd.0 Request processed in 73293us (73609us total) trans 4294974235 rc 0/0 00000100:00100000:0.0:1789760793.735901:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6946 00000100:00100000:0.0:1789760793.735911:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803963923586 time 804003617309 state 0x2 ref 0 00000001:00080000:2.0:1789760793.735993:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974235 is committed 00000100:00100000:0.0:1789760793.746370:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760793.746464:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493871872 00000100:00100000:2.0:1789760793.746474:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493871872 00000100:00100000:2.0:1789760793.746524:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 803963923586 time 804003617309 state 0x0 ref 1 00000100:00100000:2.0:1789760793.746539:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804003877725 time 804003617309 state 0x1 ref 1 00000100:00100000:2.0:1789760793.746571:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804003877725 time 804087606674 state 0x0 ref 1 00000100:00100000:2.0:1789760793.746584:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493871872:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760793.746750:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28332032, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe009 00010000:00010000:2.0:1789760793.746759:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0734d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.746774:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=804s v=(0 0) 00010000:00010000:2.0:1789760793.746778:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.746785:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0734d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.746803:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0734d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760793.791587:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28332032 -> 28336128 00010000:00010000:2.0:1789760793.791608:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0734d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.791622:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0734d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.791631:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0734d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.791639:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760793.791643:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0734d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.791653:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0734d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.791673:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0734d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.791681:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e0734d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28332032->28336127] (req 28332032->28336127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.791778:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493871872:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45192us (45413us total) trans 4294974236 rc 0/0 00000100:00100000:2.0:1789760793.791790:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6947 00000100:00100000:2.0:1789760793.791799:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804003877725 time 804087606674 state 0x2 ref 0 00000001:00080000:3.0:1789760793.792500:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974236 is committed 00000100:00100000:3.0:1789760793.795417:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760793.795677:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493872000 00000100:00100000:1.0:1789760793.795689:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493872000 00000100:00100000:1.0:1789760793.795744:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804003877725 time 804087606674 state 0x0 ref 1 00000100:00100000:1.0:1789760793.795758:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804087867090 time 804087606674 state 0x1 ref 1 00000100:00100000:1.0:1789760793.795789:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804087867090 time 804136824639 state 0x0 ref 1 00000100:00100000:1.0:1789760793.795801:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493872000:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760793.795850:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28336128, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe009 00010000:00010000:1.0:1789760793.795859:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07354 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.795874:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=804s v=(0 0) 00010000:00010000:1.0:1789760793.795879:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.795885:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07354 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.795899:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07354 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760793.828203:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28336128 -> 28340224 00010000:00010000:1.0:1789760793.828232:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07354 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.828246:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07354 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.828255:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07354 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.828262:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760793.828267:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07354 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.828277:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07354 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.828297:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07354 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.828305:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07354 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28336128->28340223] (req 28336128->28340223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.828401:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493872000:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32596us (32987us total) trans 4294974237 rc 0/0 00000100:00100000:1.0:1789760793.828412:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6948 00000100:00100000:1.0:1789760793.828422:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804087867090 time 804136824639 state 0x2 ref 0 00000001:00080000:2.0:1789760793.829340:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974237 is committed 00000100:00100000:1.0:1789760793.832756:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760793.835370:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493872128 00000100:00100000:3.0:1789760793.835386:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493872128 00000100:00100000:3.0:1789760793.835453:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804087867090 time 804136824639 state 0x0 ref 1 00000100:00100000:3.0:1789760793.835485:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804137085055 time 804136824639 state 0x1 ref 1 00000100:00100000:3.0:1789760793.835521:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804137085055 time 804176556694 state 0x0 ref 1 00000100:00100000:3.0:1789760793.835535:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493872128:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760793.835666:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28340224, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe009 00010000:00010000:3.0:1789760793.835676:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0735b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.835693:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=804s v=(0 0) 00010000:00010000:3.0:1789760793.835698:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.835704:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0735b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.835718:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0735b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760793.892707:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28340224 -> 28344320 00010000:00010000:1.0:1789760793.892724:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0735b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.892737:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0735b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.892745:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0735b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.892757:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760793.892762:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0735b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.892771:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0735b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.892788:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0735b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.892796:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e84f2a77/0x3eb3dcc0b2e0735b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28340224->28344319] (req 28340224->28344319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.892892:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493872128:12345-192.168.203.60@tcp:4:dd.0 Request processed in 57355us (60141us total) trans 4294974238 rc 0/0 00000100:00100000:1.0:1789760793.892903:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6949 00000100:00100000:1.0:1789760793.892912:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804137085055 time 804176556694 state 0x2 ref 0 00000001:00080000:2.0:1789760793.893938:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974238 is committed 00000100:00100000:3.0:1789760793.895576:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760793.895916:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493872256 00000100:00100000:0.0:1789760793.895926:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493872256 00000100:00100000:0.0:1789760793.896012:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804137085055 time 804176556694 state 0x0 ref 1 00000100:00100000:0.0:1789760793.896027:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804176817110 time 804176556694 state 0x1 ref 1 00000100:00100000:0.0:1789760793.896050:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804176817110 time 804237086262 state 0x0 ref 1 00000100:00100000:0.0:1789760793.896186:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493872256:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760793.896364:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28344320, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe009 00010000:00010000:0.0:1789760793.896374:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e07362 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.896389:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=804s v=(0 0) 00010000:00010000:0.0:1789760793.896393:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.896398:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e07362 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.896414:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e07362 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.923266:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974239 is committed 00002000:00010000:3.0:1789760793.923670:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28344320 -> 28348416 00010000:00010000:3.0:1789760793.923685:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e07362 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.923703:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e07362 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.923712:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e07362 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.923717:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760793.923720:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e07362 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.923726:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e07362 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.923741:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e07362 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.923746:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000117f56bf/0x3eb3dcc0b2e07362 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28344320->28348415] (req 28344320->28348415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.923948:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5837800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493872256:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27761us (28375us total) trans 4294974239 rc 0/0 00000100:00100000:3.0:1789760793.924002:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6950 00000100:00100000:3.0:1789760793.924016:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804176817110 time 804237086262 state 0x2 ref 0 00000100:00100000:3.0:1789760793.927357:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760793.927596:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493872384 00000100:00100000:2.0:1789760793.927607:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493872384 00000100:00100000:2.0:1789760793.927669:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804176817110 time 804237086262 state 0x0 ref 1 00000100:00100000:2.0:1789760793.927685:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804237346678 time 804237086262 state 0x1 ref 1 00000100:00100000:2.0:1789760793.927720:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804237346678 time 804268755708 state 0x0 ref 1 00000100:00100000:2.0:1789760793.927732:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493872384:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760793.927780:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28348416, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe009 00010000:00010000:2.0:1789760793.927790:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07369 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.927815:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=804s v=(0 0) 00010000:00010000:2.0:1789760793.927820:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.927827:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07369 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.927847:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07369 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.956414:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974240 is committed 00002000:00010000:3.0:1789760793.956627:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28348416 -> 28352512 00010000:00010000:3.0:1789760793.956644:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07369 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.956657:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07369 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.956667:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07369 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.956675:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760793.956679:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07369 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.956688:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07369 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.956706:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07369 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.956715:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07369 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28348416->28352511] (req 28348416->28352511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.957478:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493872384:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29073us (29455us total) trans 4294974240 rc 0/0 00000100:00100000:3.0:1789760793.957489:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6951 00000100:00100000:3.0:1789760793.957498:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804237346678 time 804268755708 state 0x2 ref 0 00000100:00100000:1.0:1789760793.959500:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760793.959593:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493872512 00000100:00100000:0.0:1789760793.959601:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493872512 00000100:00100000:0.0:1789760793.959652:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804237346678 time 804268755708 state 0x0 ref 1 00000100:00100000:0.0:1789760793.959671:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804269016124 time 804268755708 state 0x1 ref 1 00000100:00100000:0.0:1789760793.959700:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804269016124 time 804300735340 state 0x0 ref 1 00000100:00100000:0.0:1789760793.959714:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493872512:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760793.959759:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28352512, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe009 00010000:00010000:0.0:1789760793.959767:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e07370 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.959783:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=804s v=(0 0) 00010000:00010000:0.0:1789760793.959787:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.959794:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e07370 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.959819:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e07370 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.996733:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974241 is committed 00002000:00010000:3.0:1789760793.997428:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28352512 -> 28356608 00010000:00010000:3.0:1789760793.997443:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e07370 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.997456:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e07370 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.997464:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e07370 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.997471:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760793.997476:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e07370 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.997485:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e07370 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.997503:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e07370 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.997510:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006eae786c/0x3eb3dcc0b2e07370 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28352512->28356607] (req 28352512->28356607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.997625:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493872512:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37910us (38127us total) trans 4294974241 rc 0/0 00000100:00100000:3.0:1789760793.997636:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6952 00000100:00100000:3.0:1789760793.997646:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804269016124 time 804300735340 state 0x2 ref 0 00000100:00100000:0.0:1789760794.009035:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760794.009132:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493872640 00000100:00100000:1.0:1789760794.009142:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493872640 00000100:00100000:1.0:1789760794.009198:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804269016124 time 804300735340 state 0x0 ref 1 00000100:00100000:1.0:1789760794.009213:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804300995756 time 804300735340 state 0x1 ref 1 00000100:00100000:1.0:1789760794.009250:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804300995756 time 804350276943 state 0x0 ref 1 00000100:00100000:1.0:1789760794.009264:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493872640:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760794.009474:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28356608, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe009 00010000:00010000:1.0:1789760794.009484:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07377 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.009500:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=804s v=(0 0) 00010000:00010000:1.0:1789760794.009504:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.009511:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07377 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.009525:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07377 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.059318:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974242 is committed 00002000:00010000:1.0:1789760794.059472:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28356608 -> 28360704 00010000:00010000:1.0:1789760794.059486:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07377 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.059497:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07377 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.059505:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07377 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.059511:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760794.059516:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07377 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.059525:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07377 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.059543:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07377 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.059550:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07377 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28356608->28360703] (req 28356608->28360703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.059894:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4efbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493872640:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50628us (50863us total) trans 4294974242 rc 0/0 00000100:00100000:1.0:1789760794.059904:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6953 00000100:00100000:1.0:1789760794.059914:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804300995756 time 804350276943 state 0x2 ref 0 00000100:00100000:0.0:1789760794.064045:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760794.064898:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493872768 00000100:00100000:3.0:1789760794.064910:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493872768 00000100:00100000:3.0:1789760794.065150:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804300995756 time 804350276943 state 0x0 ref 1 00000100:00100000:3.0:1789760794.065167:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804350537359 time 804350276943 state 0x1 ref 1 00000100:00100000:3.0:1789760794.065198:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804350537359 time 804406233875 state 0x0 ref 1 00000100:00100000:3.0:1789760794.065220:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ef9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493872768:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760794.065271:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28360704, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe009 00010000:00010000:3.0:1789760794.065281:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0737e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.065296:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=804s v=(0 0) 00010000:00010000:3.0:1789760794.065402:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.065410:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0737e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.065426:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0737e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760794.101436:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28360704 -> 28364800 00010000:00010000:3.0:1789760794.101456:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0737e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.101474:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0737e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.101483:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0737e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.101490:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760794.101495:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0737e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.101505:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0737e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.101561:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974243 is committed 00010000:00010000:3.0:1789760794.101792:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0737e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.101800:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0737e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28360704->28364799] (req 28360704->28364799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.101894:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ef9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493872768:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36680us (37854us total) trans 4294974243 rc 0/0 00000100:00100000:3.0:1789760794.101904:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6954 00000100:00100000:3.0:1789760794.101913:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804350537359 time 804406233875 state 0x2 ref 0 00000100:00100000:3.0:1789760794.106947:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760794.107604:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493872896 00000100:00100000:2.0:1789760794.107615:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493872896 00000100:00100000:2.0:1789760794.107750:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804350537359 time 804406233875 state 0x0 ref 1 00000100:00100000:2.0:1789760794.107766:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804406494291 time 804406233875 state 0x1 ref 1 00000100:00100000:2.0:1789760794.107798:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804406494291 time 804448832683 state 0x0 ref 1 00000100:00100000:2.0:1789760794.107811:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493872896:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760794.107858:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28364800, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe009 00010000:00010000:2.0:1789760794.107868:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e07385 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.107884:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=804s v=(0 0) 00010000:00010000:2.0:1789760794.107889:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.107895:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e07385 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.107909:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e07385 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.154894:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974244 is committed 00002000:00010000:3.0:1789760794.155010:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28364800 -> 28368896 00010000:00010000:3.0:1789760794.155026:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e07385 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.155039:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e07385 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.155048:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e07385 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.155056:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760794.155060:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e07385 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.155069:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e07385 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.155100:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e07385 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.155108:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e07385 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28364800->28368895] (req 28364800->28368895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.155268:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc5835880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493872896:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47454us (48324us total) trans 4294974244 rc 0/0 00000100:00100000:3.0:1789760794.155278:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6955 00000100:00100000:3.0:1789760794.155287:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804406494291 time 804448832683 state 0x2 ref 0 00000100:00100000:0.0:1789760794.158956:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760794.159074:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493873024 00000100:00100000:3.0:1789760794.159093:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493873024 00000100:00100000:3.0:1789760794.159157:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804406494291 time 804448832683 state 0x0 ref 1 00000100:00100000:3.0:1789760794.159173:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804449093099 time 804448832683 state 0x1 ref 1 00000100:00100000:3.0:1789760794.159195:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804449093099 time 804500230722 state 0x0 ref 1 00000100:00100000:3.0:1789760794.159208:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493873024:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760794.159257:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28368896, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe009 00010000:00010000:3.0:1789760794.159265:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0738c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.159281:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=804s v=(0 0) 00010000:00010000:3.0:1789760794.159286:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.159293:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0738c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.159307:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0738c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.170123:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760794.188096:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974245 is committed 00002000:00010000:3.0:1789760794.188120:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28368896 -> 28372992 00010000:00010000:3.0:1789760794.188136:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0738c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.188149:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0738c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.188158:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0738c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.188165:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760794.188170:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0738c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.188179:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0738c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.188196:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0738c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.188203:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e0738c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28368896->28372991] (req 28368896->28372991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.188287:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493873024:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29078us (29337us total) trans 4294974245 rc 0/0 00000100:00100000:3.0:1789760794.188297:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6956 00000100:00100000:3.0:1789760794.188538:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804449093099 time 804500230722 state 0x2 ref 0 00000100:00100000:3.0:1789760794.192575:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760794.192837:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493873152 00000100:00100000:1.0:1789760794.192848:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493873152 00000100:00100000:1.0:1789760794.192903:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804449093099 time 804500230722 state 0x0 ref 1 00000100:00100000:1.0:1789760794.192919:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804500491138 time 804500230722 state 0x1 ref 1 00000100:00100000:1.0:1789760794.192950:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804500491138 time 804533985731 state 0x0 ref 1 00000100:00100000:1.0:1789760794.193002:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493873152:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760794.193062:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28372992, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe009 00010000:00010000:1.0:1789760794.193071:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07393 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.193095:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=804s v=(0 0) 00010000:00010000:1.0:1789760794.193100:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.193107:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07393 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.193121:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07393 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.224733:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974246 is committed 00002000:00010000:2.0:1789760794.224851:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28372992 -> 28377088 00010000:00010000:2.0:1789760794.224868:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07393 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.224881:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07393 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.224889:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07393 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.224896:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760794.224901:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07393 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.224911:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07393 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.224931:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07393 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.224938:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07393 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28372992->28377087] (req 28372992->28377087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.225245:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493873152:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32278us (32685us total) trans 4294974246 rc 0/0 00000100:00100000:2.0:1789760794.225260:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6957 00000100:00100000:2.0:1789760794.225270:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804500491138 time 804533985731 state 0x2 ref 0 00000100:00100000:2.0:1789760794.228645:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760794.228760:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493873280 00000100:00100000:0.0:1789760794.228770:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493873280 00000100:00100000:0.0:1789760794.228825:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804500491138 time 804533985731 state 0x0 ref 1 00000100:00100000:0.0:1789760794.228840:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804534246147 time 804533985731 state 0x1 ref 1 00000100:00100000:0.0:1789760794.228872:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804534246147 time 804569908100 state 0x0 ref 1 00000100:00100000:0.0:1789760794.228885:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493873280:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760794.228945:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28377088, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe009 00010000:00010000:0.0:1789760794.228954:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0739a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.228997:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=804s v=(0 0) 00010000:00010000:0.0:1789760794.229001:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.229007:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0739a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.229022:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0739a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760794.265268:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28377088 -> 28381184 00010000:00010000:3.0:1789760794.265291:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0739a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.265305:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0739a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.265314:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0739a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.265321:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760794.265325:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0739a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.265334:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0739a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.265354:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0739a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.265362:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e0739a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28377088->28381183] (req 28377088->28381183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.265802:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493873280:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36913us (37161us total) trans 4294974247 rc 0/0 00000100:00100000:3.0:1789760794.265812:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6958 00000100:00100000:3.0:1789760794.265822:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804534246147 time 804569908100 state 0x2 ref 0 00000001:00080000:3.0:1789760794.265870:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974247 is committed 00000100:00100000:0.0:1789760794.271530:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760794.271807:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493873408 00000100:00100000:3.0:1789760794.271816:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493873408 00000100:00100000:3.0:1789760794.271868:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804534246147 time 804569908100 state 0x0 ref 1 00000100:00100000:3.0:1789760794.271883:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804570168516 time 804569908100 state 0x1 ref 1 00000100:00100000:3.0:1789760794.271912:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804570168516 time 804612947930 state 0x0 ref 1 00000100:00100000:3.0:1789760794.271924:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4ba6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493873408:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760794.271991:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28381184, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe009 00010000:00010000:3.0:1789760794.272000:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e073a1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.272016:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=804s v=(0 0) 00010000:00010000:3.0:1789760794.272021:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.272027:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e073a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.272279:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e073a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760794.315054:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28381184 -> 28385280 00010000:00010000:3.0:1789760794.315074:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e073a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.315098:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e073a1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.315112:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e073a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.315119:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760794.315124:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e073a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.315133:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e073a1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.315168:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e073a1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.315176:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e073a1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28381184->28385279] (req 28381184->28385279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.315288:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4ba6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493873408:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43359us (43761us total) trans 4294974248 rc 0/0 00000100:00100000:3.0:1789760794.315299:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6959 00000100:00100000:3.0:1789760794.315308:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804570168516 time 804612947930 state 0x2 ref 0 00000001:00080000:2.0:1789760794.316015:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974248 is committed 00000100:00100000:2.0:1789760794.319843:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760794.319916:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493873536 00000100:00100000:3.0:1789760794.319928:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493873536 00000100:00100000:3.0:1789760794.320017:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804570168516 time 804612947930 state 0x0 ref 1 00000100:00100000:3.0:1789760794.320032:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804613208346 time 804612947930 state 0x1 ref 1 00000100:00100000:3.0:1789760794.320065:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804613208346 time 804661100511 state 0x0 ref 1 00000100:00100000:3.0:1789760794.320076:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493873536:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760794.320131:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28385280, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe009 00010000:00010000:3.0:1789760794.320149:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e073a8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.320165:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=804s v=(0 0) 00010000:00010000:3.0:1789760794.320170:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.320176:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e073a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.320190:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e073a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.356264:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974249 is committed 00002000:00010000:1.0:1789760794.356363:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28385280 -> 28389376 00010000:00010000:1.0:1789760794.356378:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e073a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.356392:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e073a8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.356401:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e073a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.356408:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760794.356413:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e073a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.356423:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e073a8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.356442:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e073a8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.356449:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2cc319/0x3eb3dcc0b2e073a8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28385280->28389375] (req 28385280->28389375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.356533:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493873536:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36453us (36696us total) trans 4294974249 rc 0/0 00000100:00100000:1.0:1789760794.356542:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6960 00000100:00100000:1.0:1789760794.356552:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804613208346 time 804661100511 state 0x2 ref 0 00000100:00100000:0.0:1789760794.368196:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760794.368330:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493873664 00000100:00100000:2.0:1789760794.368341:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493873664 00000100:00100000:2.0:1789760794.368402:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804613208346 time 804661100511 state 0x0 ref 1 00000100:00100000:2.0:1789760794.368417:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804661360927 time 804661100511 state 0x1 ref 1 00000100:00100000:2.0:1789760794.368490:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804661360927 time 804709524793 state 0x0 ref 1 00000100:00100000:2.0:1789760794.368503:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493873664:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760794.368552:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28389376, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe009 00010000:00010000:2.0:1789760794.368561:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e073af lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.368578:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=804s v=(0 0) 00010000:00010000:2.0:1789760794.368582:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.368588:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e073af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.368602:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e073af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760794.400593:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28389376 -> 28393472 00010000:00010000:1.0:1789760794.400612:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e073af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.400626:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e073af lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.400635:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e073af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.400643:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760794.400647:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e073af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.400661:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e073af lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.400687:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e073af lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.400695:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e073af lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28389376->28393471] (req 28389376->28393471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.400929:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c87a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493873664:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32422us (32785us total) trans 4294974250 rc 0/0 00000100:00100000:1.0:1789760794.400940:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6961 00000100:00100000:1.0:1789760794.400950:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804661360927 time 804709524793 state 0x2 ref 0 00000001:00080000:2.0:1789760794.401058:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974250 is committed 00000100:00100000:2.0:1789760794.406930:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760794.407289:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493873792 00000100:00100000:0.0:1789760794.407298:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493873792 00000100:00100000:0.0:1789760794.407347:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804661360927 time 804709524793 state 0x0 ref 1 00000100:00100000:0.0:1789760794.407362:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804709785209 time 804709524793 state 0x1 ref 1 00000100:00100000:0.0:1789760794.407390:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804709785209 time 804748425425 state 0x0 ref 1 00000100:00100000:0.0:1789760794.407401:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493873792:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760794.407447:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28393472, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe009 00010000:00010000:0.0:1789760794.407457:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e073b6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.407471:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=804s v=(0 0) 00010000:00010000:0.0:1789760794.407475:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.407482:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e073b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.407494:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e073b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760794.443282:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28393472 -> 28397568 00010000:00010000:3.0:1789760794.443302:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e073b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.443314:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e073b6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.443324:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e073b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.443332:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760794.443337:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e073b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.443347:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e073b6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.443365:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e073b6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.443373:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e073b6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28393472->28397567] (req 28393472->28397567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.443471:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493873792:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36065us (36544us total) trans 4294974251 rc 0/0 00000100:00100000:3.0:1789760794.443486:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6962 00000100:00100000:3.0:1789760794.443497:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804709785209 time 804748425425 state 0x2 ref 0 00000001:00080000:1.0:1789760794.444414:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974251 is committed 00000100:00100000:3.0:1789760794.447041:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760794.447629:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493873920 00000100:00100000:2.0:1789760794.447703:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493873920 00000100:00100000:2.0:1789760794.447761:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804709785209 time 804748425425 state 0x0 ref 1 00000100:00100000:2.0:1789760794.447780:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804748685841 time 804748425425 state 0x1 ref 1 00000100:00100000:2.0:1789760794.447849:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804748685841 time 804788884196 state 0x0 ref 1 00000100:00100000:2.0:1789760794.447860:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493873920:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760794.447915:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28397568, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe009 00010000:00010000:2.0:1789760794.447924:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e073bd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.447939:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=804s v=(0 0) 00010000:00010000:2.0:1789760794.447944:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.447950:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e073bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.447989:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e073bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760794.482197:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28397568 -> 28401664 00010000:00010000:2.0:1789760794.482215:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e073bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.482229:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e073bd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.482238:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e073bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.482246:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760794.482251:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e073bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.482260:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e073bd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.482278:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e073bd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.482286:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e073bd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28397568->28401663] (req 28397568->28401663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.482382:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493873920:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34517us (35345us total) trans 4294974252 rc 0/0 00000100:00100000:2.0:1789760794.482392:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6963 00000100:00100000:2.0:1789760794.482402:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804748685841 time 804788884196 state 0x2 ref 0 00000001:00080000:3.0:1789760794.487304:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974252 is committed 00000100:00100000:2.0:1789760794.487419:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760794.487510:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493874048 00000100:00100000:1.0:1789760794.487519:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493874048 00000100:00100000:1.0:1789760794.487629:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804748685841 time 804788884196 state 0x0 ref 1 00000100:00100000:1.0:1789760794.487645:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804789144612 time 804788884196 state 0x1 ref 1 00000100:00100000:1.0:1789760794.487703:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804789144612 time 804828738233 state 0x0 ref 1 00000100:00100000:1.0:1789760794.487715:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493874048:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760794.487761:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28401664, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe009 00010000:00010000:1.0:1789760794.487771:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e073c4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.487785:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=804s v=(0 0) 00010000:00010000:1.0:1789760794.487789:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.487795:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e073c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.487808:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e073c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.490198:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff962101a35f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3303:1876699519262592:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760794.490259:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760794.490331:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519262592 00000100:00100000:2.0:1789760794.490338:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519262592 00000100:00100000:2.0:1789760794.490388:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 220 00000100:00100000:2.0:1789760794.490397:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876699519262592:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760794.490421:0:6690:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528416768 pending 1703936 free 3830448128 avail 3786407936 00000100:00100000:2.0:1789760794.490462:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876699519262592:12345-0@lo:13:osp-pre-0-0.0 Request processed in 66us (202us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760794.490469:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 220 00000100:00080000:2.0:1789760794.490488:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760794.490539:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff962101a35f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3303:1876699519262592:0@lo:13:osp-pre-0-0.0 00000001:00080000:3.0:1789760794.519296:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974253 is committed 00002000:00010000:1.0:1789760794.519462:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28401664 -> 28405760 00010000:00010000:1.0:1789760794.519476:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e073c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.519489:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e073c4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.519498:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e073c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.519504:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760794.519509:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e073c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.519519:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e073c4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.519543:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e073c4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.519551:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e073c4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28401664->28405759] (req 28401664->28405759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.519879:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493874048:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32161us (32460us total) trans 4294974253 rc 0/0 00000100:00100000:1.0:1789760794.519890:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6964 00000100:00100000:1.0:1789760794.519900:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804789144612 time 804828738233 state 0x2 ref 0 00000100:00100000:1.0:1789760794.523320:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760794.523404:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493874176 00000100:00100000:2.0:1789760794.523414:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493874176 00000100:00100000:2.0:1789760794.523472:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804789144612 time 804828738233 state 0x0 ref 1 00000100:00100000:2.0:1789760794.523485:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804828998649 time 804828738233 state 0x1 ref 1 00000100:00100000:2.0:1789760794.523514:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804828998649 time 804864550548 state 0x0 ref 1 00000100:00100000:2.0:1789760794.523526:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493874176:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760794.523582:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28405760, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe009 00010000:00010000:2.0:1789760794.523590:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e073cb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.523605:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=804s v=(0 0) 00010000:00010000:2.0:1789760794.523610:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.523616:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e073cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.523629:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e073cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.552487:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974254 is committed 00002000:00010000:2.0:1789760794.552596:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28405760 -> 28409856 00010000:00010000:2.0:1789760794.552612:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e073cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.552624:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e073cb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.552632:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e073cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.552639:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760794.552644:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e073cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.552652:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e073cb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.552670:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e073cb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.552677:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ac352ff/0x3eb3dcc0b2e073cb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28405760->28409855] (req 28405760->28409855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.552803:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493874176:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29273us (29489us total) trans 4294974254 rc 0/0 00000100:00100000:2.0:1789760794.552813:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6965 00000100:00100000:2.0:1789760794.552823:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804828998649 time 804864550548 state 0x2 ref 0 00000100:00100000:3.0:1789760794.556391:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760794.556533:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493874304 00000100:00100000:0.0:1789760794.556544:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493874304 00000100:00100000:0.0:1789760794.556597:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804828998649 time 804864550548 state 0x0 ref 1 00000100:00100000:0.0:1789760794.556613:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804864810964 time 804864550548 state 0x1 ref 1 00000100:00100000:0.0:1789760794.556645:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804864810964 time 804897680651 state 0x0 ref 1 00000100:00100000:0.0:1789760794.556657:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493874304:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760794.556706:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28409856, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe009 00010000:00010000:0.0:1789760794.556715:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e073d2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.556735:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=804s v=(0 0) 00010000:00010000:0.0:1789760794.556740:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.556746:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e073d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.556759:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e073d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.582493:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974255 is committed 00002000:00010000:2.0:1789760794.582553:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28409856 -> 28413952 00010000:00010000:2.0:1789760794.582568:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e073d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.582582:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e073d2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.582591:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e073d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.582598:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760794.582603:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e073d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.582612:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e073d2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.582703:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e073d2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.582720:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e073d2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28409856->28413951] (req 28409856->28413951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.582811:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493874304:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26150us (26423us total) trans 4294974255 rc 0/0 00000100:00100000:2.0:1789760794.582824:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6966 00000100:00100000:2.0:1789760794.582833:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804864810964 time 804897680651 state 0x2 ref 0 00000100:00100000:0.0:1789760794.586917:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760794.587106:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493874432 00000100:00100000:3.0:1789760794.587116:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493874432 00000100:00100000:3.0:1789760794.587179:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804864810964 time 804897680651 state 0x0 ref 1 00000100:00100000:3.0:1789760794.587194:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804897941067 time 804897680651 state 0x1 ref 1 00000100:00100000:3.0:1789760794.587221:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804897941067 time 804928256329 state 0x0 ref 1 00000100:00100000:3.0:1789760794.587233:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c87a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493874432:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760794.587406:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28413952, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xe009 00010000:00010000:3.0:1789760794.587415:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e073d9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.587430:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=804s v=(0 0) 00010000:00010000:3.0:1789760794.587434:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.587440:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e073d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.587455:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e073d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.617951:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974256 is committed 00002000:00010000:0.0:1789760794.618405:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28413952 -> 28418048 00002000:00010000:0.0:1789760794.618410:0:13038:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760793 -> 1789760794 00002000:00010000:0.0:1789760794.618412:0:13038:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760793 -> 1789760794 00010000:00010000:0.0:1789760794.618425:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e073d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.618443:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e073d9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.618453:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e073d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.618460:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdac00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760794.618465:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e073d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.618475:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e073d9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.618494:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e073d9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.618502:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093621ceb/0x3eb3dcc0b2e073d9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28413952->28418047] (req 28413952->28418047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.618870:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c87a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493874432:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31632us (31955us total) trans 4294974256 rc 0/0 00000100:00100000:0.0:1789760794.618881:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6967 00000100:00100000:0.0:1789760794.618891:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804897941067 time 804928256329 state 0x2 ref 0 00000100:00100000:1.0:1789760794.630876:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760794.632236:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493874560 00000100:00100000:3.0:1789760794.632247:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493874560 00000100:00100000:3.0:1789760794.632305:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804897941067 time 804928256329 state 0x0 ref 1 00000100:00100000:3.0:1789760794.632320:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804928516745 time 804928256329 state 0x1 ref 1 00000100:00100000:3.0:1789760794.632353:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804928516745 time 804973388529 state 0x0 ref 1 00000100:00100000:3.0:1789760794.632366:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493874560:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760794.632416:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28418048, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe009 00010000:00010000:3.0:1789760794.632425:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e073e0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.632447:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=804s v=(0 0) 00010000:00010000:3.0:1789760794.632451:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.632457:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e073e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.632481:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e073e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.663480:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974257 is committed 00002000:00010000:0.0:1789760794.663683:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28418048 -> 28422144 00010000:00010000:0.0:1789760794.663699:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e073e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.663712:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e073e0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.663722:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e073e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.663729:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760794.663734:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e073e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.663792:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e073e0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.663813:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e073e0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.663820:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e073e0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28418048->28422143] (req 28418048->28422143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.663910:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493874560:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31542us (33039us total) trans 4294974257 rc 0/0 00000100:00100000:0.0:1789760794.663920:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6968 00000100:00100000:0.0:1789760794.663929:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804928516745 time 804973388529 state 0x2 ref 0 00000100:00100000:0.0:1789760794.669755:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760794.671532:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493874688 00000100:00100000:2.0:1789760794.671544:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493874688 00000100:00100000:2.0:1789760794.671607:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804928516745 time 804973388529 state 0x0 ref 1 00000100:00100000:2.0:1789760794.671624:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804973648945 time 804973388529 state 0x1 ref 1 00000100:00100000:2.0:1789760794.671699:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804973648945 time 805012734604 state 0x0 ref 1 00000100:00100000:2.0:1789760794.671713:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fadb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493874688:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760794.671761:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28422144, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe009 00010000:00010000:2.0:1789760794.671769:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e073e7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.671785:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=805s v=(0 0) 00010000:00010000:2.0:1789760794.671789:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.671796:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e073e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.671809:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e073e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.710568:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974258 is committed 00002000:00010000:2.0:1789760794.710655:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28422144 -> 28426240 00010000:00010000:2.0:1789760794.710669:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e073e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.710680:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e073e7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.710689:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e073e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.710697:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760794.710702:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e073e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.710712:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e073e7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.710731:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e073e7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.710739:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e073e7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28422144->28426239] (req 28422144->28426239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.710882:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fadb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493874688:12345-192.168.203.60@tcp:4:dd.0 Request processed in 39167us (41131us total) trans 4294974258 rc 0/0 00000100:00100000:2.0:1789760794.710893:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6969 00000100:00100000:2.0:1789760794.710903:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804973648945 time 805012734604 state 0x2 ref 0 00000100:00100000:2.0:1789760794.714070:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760794.714225:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493874816 00000100:00100000:0.0:1789760794.714235:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493874816 00000100:00100000:0.0:1789760794.714287:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 804973648945 time 805012734604 state 0x0 ref 1 00000100:00100000:0.0:1789760794.714302:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805012995020 time 805012734604 state 0x1 ref 1 00000100:00100000:0.0:1789760794.714369:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805012995020 time 805055403957 state 0x0 ref 1 00000100:00100000:0.0:1789760794.714383:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493874816:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760794.714437:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28426240, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe009 00010000:00010000:0.0:1789760794.714446:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e073ee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.714461:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=805s v=(0 0) 00010000:00010000:0.0:1789760794.714465:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.714469:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e073ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.714480:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e073ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.747735:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974259 is committed 00002000:00010000:0.0:1789760794.747771:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28426240 -> 28430336 00010000:00010000:0.0:1789760794.747791:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e073ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.747809:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e073ee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.747817:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e073ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.747824:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760794.747828:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e073ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.747836:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e073ee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.747852:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e073ee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.747859:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e073ee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28426240->28430335] (req 28426240->28430335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.747955:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493874816:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33568us (33888us total) trans 4294974259 rc 0/0 00000100:00100000:0.0:1789760794.748002:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6970 00000100:00100000:0.0:1789760794.748011:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805012995020 time 805055403957 state 0x2 ref 0 00000100:00100000:1.0:1789760794.754342:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760794.754448:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493874944 00000100:00100000:0.0:1789760794.754459:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493874944 00000100:00100000:0.0:1789760794.754517:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805012995020 time 805055403957 state 0x0 ref 1 00000100:00100000:0.0:1789760794.754532:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805055664373 time 805055403957 state 0x1 ref 1 00000100:00100000:0.0:1789760794.754565:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805055664373 time 805095600442 state 0x0 ref 1 00000100:00100000:0.0:1789760794.754577:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493874944:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760794.754623:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28430336, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe009 00010000:00010000:0.0:1789760794.754632:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e073f5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.754647:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=805s v=(0 0) 00010000:00010000:0.0:1789760794.754651:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.754658:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e073f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.754671:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e073f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.794708:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974260 is committed 00002000:00010000:0.0:1789760794.794835:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28430336 -> 28434432 00010000:00010000:0.0:1789760794.794851:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e073f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.794863:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e073f5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.794878:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e073f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.794885:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760794.794890:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e073f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.794899:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e073f5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.794919:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e073f5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.794927:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e073f5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28430336->28434431] (req 28430336->28434431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.795063:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493874944:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40482us (40725us total) trans 4294974260 rc 0/0 00000100:00100000:0.0:1789760794.795073:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6971 00000100:00100000:0.0:1789760794.795094:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805055664373 time 805095600442 state 0x2 ref 0 00000100:00100000:2.0:1789760794.798731:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760794.798819:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493875072 00000100:00100000:0.0:1789760794.798828:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493875072 00000100:00100000:0.0:1789760794.798881:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805055664373 time 805095600442 state 0x0 ref 1 00000100:00100000:0.0:1789760794.798896:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805095860858 time 805095600442 state 0x1 ref 1 00000100:00100000:0.0:1789760794.798932:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805095860858 time 805139966953 state 0x0 ref 1 00000100:00100000:0.0:1789760794.798944:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a34e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493875072:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760794.799023:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28434432, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe009 00010000:00010000:0.0:1789760794.799033:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e073fc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.799048:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=805s v=(0 0) 00010000:00010000:0.0:1789760794.799052:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.799059:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e073fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.799073:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e073fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760794.834026:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28434432 -> 28438528 00010000:00010000:0.0:1789760794.834046:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e073fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.834060:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e073fc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.834069:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e073fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.834076:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760794.834094:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e073fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.834112:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e073fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.834130:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e073fc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.834146:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e073fc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28434432->28438527] (req 28434432->28438527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.834228:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a34e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493875072:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35281us (35501us total) trans 4294974261 rc 0/0 00000100:00100000:0.0:1789760794.834238:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6972 00000100:00100000:0.0:1789760794.834248:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805095860858 time 805139966953 state 0x2 ref 0 00000001:00080000:1.0:1789760794.835321:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974261 is committed 00000100:00100000:2.0:1789760794.856766:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760794.856868:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493875200 00000100:00100000:1.0:1789760794.856878:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493875200 00000100:00100000:1.0:1789760794.856921:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805095860858 time 805139966953 state 0x0 ref 1 00000100:00100000:1.0:1789760794.856934:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805140227369 time 805139966953 state 0x1 ref 1 00000100:00100000:1.0:1789760794.856986:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805140227369 time 805197994936 state 0x0 ref 1 00000100:00100000:1.0:1789760794.857000:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff962101a37480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493875200:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760794.857186:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28438528, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe009 00010000:00010000:0.0:1789760794.857197:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e07403 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.857212:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=805s v=(0 0) 00010000:00010000:0.0:1789760794.857216:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.857222:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e07403 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.857235:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e07403 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760794.891095:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28438528 -> 28442624 00010000:00010000:2.0:1789760794.891117:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e07403 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.891137:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e07403 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.891147:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e07403 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.891154:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760794.891159:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e07403 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.891168:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e07403 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.891191:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e07403 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.891198:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e07403 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28438528->28442623] (req 28438528->28442623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.891299:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff962101a37480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493875200:12345-192.168.203.60@tcp:4:dd.0 Request processed in 34296us (34537us total) trans 4294974262 rc 0/0 00000100:00100000:2.0:1789760794.891309:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6973 00000100:00100000:2.0:1789760794.891317:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805140227369 time 805197994936 state 0x2 ref 0 00000001:00080000:3.0:1789760794.891793:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974262 is committed 00000100:00100000:2.0:1789760794.895114:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760794.895221:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493875328 00000100:00100000:1.0:1789760794.895232:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493875328 00000100:00100000:1.0:1789760794.895297:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805140227369 time 805197994936 state 0x0 ref 1 00000100:00100000:1.0:1789760794.895316:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805198255352 time 805197994936 state 0x1 ref 1 00000100:00100000:1.0:1789760794.895338:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805198255352 time 805236374297 state 0x0 ref 1 00000100:00100000:1.0:1789760794.895350:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffc14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493875328:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760794.895409:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28442624, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe009 00010000:00010000:1.0:1789760794.895419:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0740a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.895437:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=805s v=(0 0) 00010000:00010000:1.0:1789760794.895441:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.895447:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0740a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.895466:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0740a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.927487:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974263 is committed 00002000:00010000:3.0:1789760794.928221:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28442624 -> 28446720 00010000:00010000:3.0:1789760794.928236:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0740a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.928249:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0740a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.928259:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0740a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.928266:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760794.928271:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0740a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.928281:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0740a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.928306:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0740a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.928314:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e0740a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28442624->28446719] (req 28442624->28446719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.928443:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffc14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493875328:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33089us (33335us total) trans 4294974263 rc 0/0 00000100:00100000:3.0:1789760794.928454:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6974 00000100:00100000:3.0:1789760794.928464:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805198255352 time 805236374297 state 0x2 ref 0 00000100:00100000:1.0:1789760794.933650:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760794.933745:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493875456 00000100:00100000:0.0:1789760794.933754:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493875456 00000100:00100000:0.0:1789760794.933805:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805198255352 time 805236374297 state 0x0 ref 1 00000100:00100000:0.0:1789760794.933819:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805236634713 time 805236374297 state 0x1 ref 1 00000100:00100000:0.0:1789760794.933853:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805236634713 time 805274888259 state 0x0 ref 1 00000100:00100000:0.0:1789760794.933865:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493875456:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760794.933924:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28446720, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe009 00010000:00010000:0.0:1789760794.933933:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07411 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.933950:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=805s v=(0 0) 00010000:00010000:0.0:1789760794.933954:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.933983:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07411 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28446720->28450815] (req 28446720->28450815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.933998:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07411 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28446720->28450815] (req 28446720->28450815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.964175:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974264 is committed 00002000:00010000:1.0:1789760794.964331:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28446720 -> 28450816 00010000:00010000:1.0:1789760794.964347:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07411 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28446720->28450815] (req 28446720->28450815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.964360:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07411 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28446720->28450815] (req 28446720->28450815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.964368:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07411 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28446720->28450815] (req 28446720->28450815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.964376:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760794.964381:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07411 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28446720->28450815] (req 28446720->28450815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.964390:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07411 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28446720->28450815] (req 28446720->28450815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.964411:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07411 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28446720->28450815] (req 28446720->28450815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.964419:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07411 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28446720->28450815] (req 28446720->28450815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.964513:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493875456:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30644us (30866us total) trans 4294974264 rc 0/0 00000100:00100000:1.0:1789760794.964522:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6975 00000100:00100000:1.0:1789760794.964532:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805236634713 time 805274888259 state 0x2 ref 0 00000100:00100000:1.0:1789760794.968248:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760794.968352:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493875584 00000100:00100000:0.0:1789760794.968361:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493875584 00000100:00100000:0.0:1789760794.968415:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805236634713 time 805274888259 state 0x0 ref 1 00000100:00100000:0.0:1789760794.968431:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805275148675 time 805274888259 state 0x1 ref 1 00000100:00100000:0.0:1789760794.968505:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805275148675 time 805309539500 state 0x0 ref 1 00000100:00100000:0.0:1789760794.968517:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493875584:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760794.968566:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28450816, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe009 00010000:00010000:0.0:1789760794.968574:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e07418 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.968589:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=805s v=(0 0) 00010000:00010000:0.0:1789760794.968593:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.968599:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e07418 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28450816->28454911] (req 28450816->28454911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.968613:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e07418 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28450816->28454911] (req 28450816->28454911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760794.993600:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28450816 -> 28454912 00010000:00010000:0.0:1789760794.993620:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e07418 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28450816->28454911] (req 28450816->28454911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.993634:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e07418 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28450816->28454911] (req 28450816->28454911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.993643:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e07418 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28450816->28454911] (req 28450816->28454911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.993650:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760794.993655:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e07418 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28450816->28454911] (req 28450816->28454911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.993665:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e07418 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28450816->28454911] (req 28450816->28454911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.993683:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e07418 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28450816->28454911] (req 28450816->28454911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.993690:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e07418 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28450816->28454911] (req 28450816->28454911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.993777:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493875584:12345-192.168.203.60@tcp:4:dd.0 Request processed in 25257us (25534us total) trans 4294974265 rc 0/0 00000100:00100000:0.0:1789760794.993787:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6976 00000100:00100000:0.0:1789760794.993796:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805275148675 time 805309539500 state 0x2 ref 0 00000001:00080000:2.0:1789760794.994310:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974265 is committed 00000100:00100000:2.0:1789760794.997625:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760794.997737:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493875712 00000100:00100000:1.0:1789760794.997746:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493875712 00000100:00100000:1.0:1789760794.997867:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805275148675 time 805309539500 state 0x0 ref 1 00000100:00100000:1.0:1789760794.997883:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805309799916 time 805309539500 state 0x1 ref 1 00000100:00100000:1.0:1789760794.997948:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805309799916 time 805338983196 state 0x0 ref 1 00000100:00100000:1.0:1789760794.997979:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8794700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493875712:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760794.998030:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28454912, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe009 00010000:00010000:1.0:1789760794.998038:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0741f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.998053:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=805s v=(0 0) 00010000:00010000:1.0:1789760794.998058:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.998064:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0741f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28454912->28459007] (req 28454912->28459007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.998093:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0741f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28454912->28459007] (req 28454912->28459007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.005181:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4363480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3305:1876699519262848:0@lo:400:kworker.0 00000100:00100000:0.0:1789760795.005252:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760795.005309:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc1d08e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519263360:0@lo:400:kworker.0 00000100:00100000:0.0:1789760795.005335:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760795.005362:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789760795.005368:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4362d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519262976:0@lo:400:kworker.0 00000100:00100000:0.0:1789760795.005392:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760795.005425:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519262848 00000100:00100000:0.0:1789760795.005433:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519262848 00000100:00100000:1.0:1789760795.005678:0:3303:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4360700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3303:1876699519263104:0@lo:400:kworker.0 00000100:00100000:1.0:1789760795.005709:0:3303:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760795.005755:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc4361500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3304:1876699519262720:0@lo:400:kworker.0 00000100:00100000:2.0:1789760795.005789:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760795.005820:0:3304:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9620c89fb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3304:1876699519263232:0@lo:400:kworker.0 00000100:00100000:3.0:1789760795.005988:0:5840:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519262720 00000100:00100000:3.0:1789760795.005993:0:5840:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519262720 00000100:00100000:3.0:1789760795.006024:0:5840:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 420 00000100:00100000:3.0:1789760795.006033:0:5840:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7d06450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3304:x1876699519262720:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760795.006105:0:5840:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7d06450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963+19:3304:x1876699519262720:12345-0@lo:400:kworker.0 Request processed in 73us (315us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760795.006113:0:5840:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 420 00000100:00100000:0.0:1789760795.006493:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519263104 00000100:00100000:0.0:1789760795.006499:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519263104 00000100:00100000:0.0:1789760795.006544:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 534 00000100:00100000:0.0:1789760795.006550:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3305:x1876699519262848:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760795.006585:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb461c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3305:x1876699519262848:12345-0@lo:400:kworker.0 Request processed in 36us (1334us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760795.006591:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 534 00000100:00100000:0.0:1789760795.006602:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 535 00000100:00100000:0.0:1789760795.006607:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876699519263104:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760795.006633:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876699519263104:12345-0@lo:400:kworker.0 Request processed in 26us (923us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760795.006638:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 535 00000100:00100000:0.0:1789760795.006653:0:10475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519263360 00000100:00100000:0.0:1789760795.006656:0:10475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519263360 00000100:00100000:0.0:1789760795.006679:0:10475:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519262976 00000100:00100000:0.0:1789760795.006681:0:10475:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519262976 00000100:00100000:0.0:1789760795.006696:0:10475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 190 00000100:00100000:0.0:1789760795.006701:0:10475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519263360:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760795.006727:0:10475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519263360:12345-0@lo:400:kworker.0 Request processed in 27us (1391us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760795.006732:0:10475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 190 00000100:00100000:0.0:1789760795.006741:0:10475:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 191 00000100:00100000:0.0:1789760795.006746:0:10475:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9621016d3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519262976:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760795.006767:0:10475:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9621016d3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519262976:12345-0@lo:400:kworker.0 Request processed in 21us (1374us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760795.006771:0:10475:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 191 00000100:00100000:0.0:1789760795.006999:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4362d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519262976:0@lo:400:kworker.0 00000100:00100000:0.0:1789760795.007011:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4363480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3305:1876699519262848:0@lo:400:kworker.0 00000100:00100000:0.0:1789760795.007019:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc1d08e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519263360:0@lo:400:kworker.0 00000100:00100000:0.0:1789760795.007047:0:3303:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4360700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3303:1876699519263104:0@lo:400:kworker.0 00000100:00080000:1.0:1789760795.007245:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789760795.007250:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760795.007544:0:3304:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760795.007591:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc4361500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac8d50d2-24d2-46b6-a35a-35cbd1ccf963:3304:1876699519262720:0@lo:400:kworker.0 00000100:00100000:0.0:1789760795.007793:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519263232 00000100:00100000:0.0:1789760795.007798:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519263232 00000100:00100000:0.0:1789760795.007846:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 536 00000100:00100000:0.0:1789760795.007854:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8796a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3304:x1876699519263232:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760795.007904:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8796a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3304:x1876699519263232:12345-0@lo:400:kworker.0 Request processed in 50us (359us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760795.007910:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 536 00000100:00100000:0.0:1789760795.007937:0:3304:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9620c89fb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3304:1876699519263232:0@lo:400:kworker.0 00000100:00080000:1.0:1789760795.008049:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:1.0:1789760795.061873:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974266 is committed 00002000:00010000:1.0:1789760795.065719:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28454912 -> 28459008 00010000:00010000:1.0:1789760795.065746:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0741f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28454912->28459007] (req 28454912->28459007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.065759:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0741f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28454912->28459007] (req 28454912->28459007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.065769:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0741f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28454912->28459007] (req 28454912->28459007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.065776:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760795.065781:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0741f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28454912->28459007] (req 28454912->28459007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.065789:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0741f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28454912->28459007] (req 28454912->28459007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.065809:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0741f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28454912->28459007] (req 28454912->28459007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.065817:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e0741f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28454912->28459007] (req 28454912->28459007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.065928:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8794700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493875712:12345-192.168.203.60@tcp:4:dd.0 Request processed in 67964us (68308us total) trans 4294974266 rc 0/0 00000100:00100000:1.0:1789760795.065938:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6977 00000100:00100000:1.0:1789760795.065948:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805309799916 time 805338983196 state 0x2 ref 0 00000100:00100000:1.0:1789760795.069345:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760795.070136:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493875840 00000100:00100000:3.0:1789760795.070146:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493875840 00000100:00100000:3.0:1789760795.070214:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805309799916 time 805338983196 state 0x0 ref 1 00000100:00100000:3.0:1789760795.070229:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805339243612 time 805338983196 state 0x1 ref 1 00000100:00100000:3.0:1789760795.070275:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805339243612 time 805411310791 state 0x0 ref 1 00000100:00100000:3.0:1789760795.070293:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493875840:12345-192.168.203.60@tcp:4:dd.0 00000100:00080000:1.0:1789760795.070298:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789760795.070442:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28459008, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe009 00010000:00010000:3.0:1789760795.070452:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07426 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.070469:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=805s v=(0 0) 00010000:00010000:3.0:1789760795.070473:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.070480:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07426 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28459008->28463103] (req 28459008->28463103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.070494:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07426 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28459008->28463103] (req 28459008->28463103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760795.111219:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28459008 -> 28463104 00010000:00010000:1.0:1789760795.111239:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07426 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28459008->28463103] (req 28459008->28463103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.111253:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07426 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28459008->28463103] (req 28459008->28463103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.111263:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07426 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28459008->28463103] (req 28459008->28463103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.111270:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760795.111275:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07426 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28459008->28463103] (req 28459008->28463103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.111284:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07426 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28459008->28463103] (req 28459008->28463103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.111302:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07426 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28459008->28463103] (req 28459008->28463103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.111309:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e07426 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28459008->28463103] (req 28459008->28463103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.111411:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493875840:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41117us (42069us total) trans 4294974267 rc 0/0 00000100:00100000:1.0:1789760795.111421:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6978 00000100:00100000:1.0:1789760795.111432:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805339243612 time 805411310791 state 0x2 ref 0 00000001:00080000:2.0:1789760795.113182:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974267 is committed 00000100:00100000:2.0:1789760795.119373:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760795.119441:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493875968 00000100:00100000:0.0:1789760795.119452:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493875968 00000100:00100000:0.0:1789760795.119518:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805339243612 time 805411310791 state 0x0 ref 1 00000100:00100000:0.0:1789760795.119536:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805411571207 time 805411310791 state 0x1 ref 1 00000100:00100000:0.0:1789760795.119570:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805411571207 time 805460605506 state 0x0 ref 1 00000100:00100000:0.0:1789760795.119583:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493875968:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760795.119631:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28463104, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe009 00010000:00010000:0.0:1789760795.119640:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0742d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.119660:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=805s v=(0 0) 00010000:00010000:0.0:1789760795.119665:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.119672:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0742d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28463104->28467199] (req 28463104->28467199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.119686:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0742d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28463104->28467199] (req 28463104->28467199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.161370:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974268 is committed 00002000:00010000:1.0:1789760795.161498:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28463104 -> 28467200 00010000:00010000:1.0:1789760795.161514:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0742d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28463104->28467199] (req 28463104->28467199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.161527:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0742d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28463104->28467199] (req 28463104->28467199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.161536:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0742d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28463104->28467199] (req 28463104->28467199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.161544:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760795.161549:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0742d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28463104->28467199] (req 28463104->28467199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.161558:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0742d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28463104->28467199] (req 28463104->28467199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.161576:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0742d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28463104->28467199] (req 28463104->28467199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.161584:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e0742d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28463104->28467199] (req 28463104->28467199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.161679:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493875968:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42094us (42312us total) trans 4294974268 rc 0/0 00000100:00100000:1.0:1789760795.161690:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6979 00000100:00100000:1.0:1789760795.161700:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805411571207 time 805460605506 state 0x2 ref 0 00000100:00100000:3.0:1789760795.165598:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760795.165763:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493876096 00000100:00100000:2.0:1789760795.165778:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493876096 00000100:00100000:2.0:1789760795.165865:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805411571207 time 805460605506 state 0x0 ref 1 00000100:00100000:2.0:1789760795.165884:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805460865922 time 805460605506 state 0x1 ref 1 00000100:00100000:3.0:1789760795.166323:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805460865922 time 805507354521 state 0x0 ref 1 00000100:00100000:3.0:1789760795.166351:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb15850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493876096:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760795.166444:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28467200, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe009 00010000:00010000:3.0:1789760795.166461:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07434 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.166481:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=805s v=(0 0) 00010000:00010000:3.0:1789760795.166486:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.166501:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07434 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28467200->28471295] (req 28467200->28471295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.166519:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07434 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28467200->28471295] (req 28467200->28471295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.195682:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760795.237204:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974269 is committed 00002000:00010000:3.0:1789760795.237339:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28467200 -> 28471296 00010000:00010000:3.0:1789760795.237354:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07434 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28467200->28471295] (req 28467200->28471295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.237371:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07434 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28467200->28471295] (req 28467200->28471295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.237381:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07434 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28467200->28471295] (req 28467200->28471295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.237389:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760795.237395:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07434 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28467200->28471295] (req 28467200->28471295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.237404:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07434 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28467200->28471295] (req 28467200->28471295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.237427:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07434 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28467200->28471295] (req 28467200->28471295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.237434:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07434 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28467200->28471295] (req 28467200->28471295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.237534:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb15850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493876096:12345-192.168.203.60@tcp:4:dd.0 Request processed in 71193us (71948us total) trans 4294974269 rc 0/0 00000100:00100000:3.0:1789760795.237544:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6980 00000100:00100000:3.0:1789760795.237554:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805460865922 time 805507354521 state 0x2 ref 0 00000100:00100000:1.0:1789760795.253191:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760795.253332:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493876224 00000100:00100000:2.0:1789760795.253342:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493876224 00000100:00100000:2.0:1789760795.253397:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805460865922 time 805507354521 state 0x0 ref 1 00000100:00100000:2.0:1789760795.253413:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805507614937 time 805507354521 state 0x1 ref 1 00000100:00100000:2.0:1789760795.253441:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805507614937 time 805594476339 state 0x0 ref 1 00000100:00100000:2.0:1789760795.253454:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493876224:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760795.253500:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28471296, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe009 00010000:00010000:2.0:1789760795.253509:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0743b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.253525:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=805s v=(0 0) 00010000:00010000:2.0:1789760795.253529:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.253536:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0743b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28471296->28475391] (req 28471296->28475391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.253549:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0743b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28471296->28475391] (req 28471296->28475391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.344973:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974270 is committed 00002000:00010000:3.0:1789760795.346946:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28471296 -> 28475392 00010000:00010000:3.0:1789760795.346991:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0743b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28471296->28475391] (req 28471296->28475391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.347007:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0743b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28471296->28475391] (req 28471296->28475391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.347017:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0743b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28471296->28475391] (req 28471296->28475391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.347024:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760795.347030:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0743b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28471296->28475391] (req 28471296->28475391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.347039:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0743b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28471296->28475391] (req 28471296->28475391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.347059:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0743b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28471296->28475391] (req 28471296->28475391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.347072:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0743b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28471296->28475391] (req 28471296->28475391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.347195:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493876224:12345-192.168.203.60@tcp:4:dd.0 Request processed in 93739us (94007us total) trans 4294974270 rc 0/0 00000100:00100000:3.0:1789760795.347206:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6981 00000100:00100000:3.0:1789760795.347215:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805507614937 time 805594476339 state 0x2 ref 0 00000100:00100000:2.0:1789760795.350506:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760795.350607:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493876352 00000100:00100000:1.0:1789760795.350619:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493876352 00000100:00100000:1.0:1789760795.350684:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805507614937 time 805594476339 state 0x0 ref 1 00000100:00100000:1.0:1789760795.350699:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805594736755 time 805594476339 state 0x1 ref 1 00000100:00100000:1.0:1789760795.350730:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805594736755 time 805691765725 state 0x0 ref 1 00000100:00100000:1.0:1789760795.350743:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d88e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493876352:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760795.350797:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28475392, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe009 00010000:00010000:1.0:1789760795.350806:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07442 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.350822:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=805s v=(0 0) 00010000:00010000:1.0:1789760795.350827:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.350833:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07442 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28475392->28479487] (req 28475392->28479487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.350853:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07442 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28475392->28479487] (req 28475392->28479487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760795.480413:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28475392 -> 28479488 00010000:00010000:1.0:1789760795.480431:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07442 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28475392->28479487] (req 28475392->28479487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.480445:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07442 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28475392->28479487] (req 28475392->28479487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.480455:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07442 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28475392->28479487] (req 28475392->28479487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.480462:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760795.480467:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07442 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28475392->28479487] (req 28475392->28479487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.480476:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07442 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28475392->28479487] (req 28475392->28479487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.480494:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07442 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28475392->28479487] (req 28475392->28479487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.480502:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07442 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28475392->28479487] (req 28475392->28479487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.480598:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d88e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493876352:12345-192.168.203.60@tcp:4:dd.0 Request processed in 129852us (130099us total) trans 4294974271 rc 0/0 00000100:00100000:1.0:1789760795.480609:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6982 00000100:00100000:1.0:1789760795.480620:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805594736755 time 805691765725 state 0x2 ref 0 00000001:00080000:2.0:1789760795.481450:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974271 is committed 00000100:00100000:2.0:1789760795.483578:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760795.483677:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493876480 00000100:00100000:1.0:1789760795.483687:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493876480 00000100:00100000:1.0:1789760795.483743:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805594736755 time 805691765725 state 0x0 ref 1 00000100:00100000:1.0:1789760795.483758:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805692026141 time 805691765725 state 0x1 ref 1 00000100:00100000:1.0:1789760795.483789:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805692026141 time 805824825260 state 0x0 ref 1 00000100:00100000:1.0:1789760795.483802:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493876480:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760795.483849:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28479488, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe009 00010000:00010000:1.0:1789760795.483857:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07449 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.483872:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=805s v=(0 0) 00010000:00010000:1.0:1789760795.483876:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.483883:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07449 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28479488->28483583] (req 28479488->28483583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.483895:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07449 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28479488->28483583] (req 28479488->28483583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.510780:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974272 is committed 00002000:00010000:1.0:1789760795.510923:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28479488 -> 28483584 00010000:00010000:1.0:1789760795.510937:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07449 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28479488->28483583] (req 28479488->28483583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.510951:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07449 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28479488->28483583] (req 28479488->28483583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.510988:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07449 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28479488->28483583] (req 28479488->28483583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.510997:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760795.511002:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07449 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28479488->28483583] (req 28479488->28483583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.511011:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07449 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28479488->28483583] (req 28479488->28483583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.511031:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07449 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28479488->28483583] (req 28479488->28483583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.511038:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e07449 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28479488->28483583] (req 28479488->28483583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.511371:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493876480:12345-192.168.203.60@tcp:4:dd.0 Request processed in 27566us (27796us total) trans 4294974272 rc 0/0 00000100:00100000:1.0:1789760795.511386:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6983 00000100:00100000:1.0:1789760795.511397:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805692026141 time 805824825260 state 0x2 ref 0 00000100:00100000:1.0:1789760795.514567:0:3305:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760795.514592:0:3305:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff961fc48dd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519263488:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760795.514658:0:3305:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760795.514908:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760795.515018:0:6690:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699519263488 00000100:00100000:2.0:1789760795.515026:0:6690:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699519263488 00000100:00100000:3.0:1789760795.515047:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493876608 00000100:00100000:3.0:1789760795.515054:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493876608 00000100:00100000:2.0:1789760795.515071:0:6690:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 221 00000100:00100000:2.0:1789760795.515126:0:6690:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519263488:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760795.515157:0:6690:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760795.515266:0:6690:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3305:x1876699519263488:12345-0@lo:13:osp-pre-1-0.0 Request processed in 141us (607us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760795.515309:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805692026141 time 805824825260 state 0x0 ref 1 00000100:00100000:1.0:1789760795.515318:0:3305:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff961fc48dd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3305:1876699519263488:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760795.515326:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805825085676 time 805824825260 state 0x1 ref 1 00000100:00100000:3.0:1789760795.515351:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805825085676 time 805856387378 state 0x0 ref 1 00000100:00100000:3.0:1789760795.515362:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493876608:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760795.515401:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28483584, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe009 00010000:00010000:3.0:1789760795.515409:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07450 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.515424:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=805s v=(0 0) 00010000:00010000:3.0:1789760795.515429:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.515436:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07450 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28483584->28487679] (req 28483584->28487679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.515455:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07450 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28483584->28487679] (req 28483584->28487679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.515533:0:6690:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 221 00002000:00010000:0.0:1789760795.563300:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28483584 -> 28487680 00010000:00010000:0.0:1789760795.563321:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07450 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28483584->28487679] (req 28483584->28487679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.563335:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07450 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28483584->28487679] (req 28483584->28487679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.563345:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07450 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28483584->28487679] (req 28483584->28487679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.563355:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdae00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760795.563359:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07450 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28483584->28487679] (req 28483584->28487679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.563368:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07450 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28483584->28487679] (req 28483584->28487679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.563393:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07450 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28483584->28487679] (req 28483584->28487679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.563402:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e07450 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28483584->28487679] (req 28483584->28487679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.563495:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493876608:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48129us (48583us total) trans 4294974273 rc 0/0 00000100:00100000:0.0:1789760795.563505:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6984 00000100:00100000:0.0:1789760795.563514:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805825085676 time 805856387378 state 0x2 ref 0 00000001:00080000:1.0:1789760795.564310:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974273 is committed 00000100:00100000:2.0:1789760795.567365:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760795.567449:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493876736 00000100:00100000:3.0:1789760795.567459:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493876736 00000100:00100000:3.0:1789760795.567516:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805825085676 time 805856387378 state 0x0 ref 1 00000100:00100000:3.0:1789760795.567532:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805856647794 time 805856387378 state 0x1 ref 1 00000100:00100000:3.0:1789760795.567634:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805856647794 time 805908667571 state 0x0 ref 1 00000100:00100000:3.0:1789760795.567649:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d89c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493876736:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760795.567700:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28487680, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe009 00010000:00010000:3.0:1789760795.567708:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07457 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.567723:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=805s v=(0 0) 00010000:00010000:3.0:1789760795.567727:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.567733:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07457 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28487680->28491775] (req 28487680->28491775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.567746:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07457 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28487680->28491775] (req 28487680->28491775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760795.603565:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28487680 -> 28491776 00010000:00010000:2.0:1789760795.603584:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07457 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28487680->28491775] (req 28487680->28491775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.603598:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07457 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28487680->28491775] (req 28487680->28491775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.603606:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07457 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28487680->28491775] (req 28487680->28491775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.603616:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760795.603621:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07457 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28487680->28491775] (req 28487680->28491775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.603630:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07457 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28487680->28491775] (req 28487680->28491775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.603655:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07457 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28487680->28491775] (req 28487680->28491775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.603662:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07457 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28487680->28491775] (req 28487680->28491775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.603757:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d89c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493876736:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36107us (36396us total) trans 4294974274 rc 0/0 00000100:00100000:2.0:1789760795.603769:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6985 00000100:00100000:2.0:1789760795.603779:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805856647794 time 805908667571 state 0x2 ref 0 00000001:00080000:1.0:1789760795.604386:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974274 is committed 00000100:00100000:2.0:1789760795.613026:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760795.613133:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493876864 00000100:00100000:3.0:1789760795.613145:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493876864 00000100:00100000:3.0:1789760795.613200:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805856647794 time 805908667571 state 0x0 ref 1 00000100:00100000:3.0:1789760795.613216:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805908927987 time 805908667571 state 0x1 ref 1 00000100:00100000:3.0:1789760795.613236:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805908927987 time 805954272429 state 0x0 ref 1 00000100:00100000:3.0:1789760795.613247:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493876864:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760795.613298:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28491776, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xe009 00010000:00010000:3.0:1789760795.613307:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0745e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.613323:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=805s v=(0 0) 00010000:00010000:3.0:1789760795.613328:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.613335:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0745e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28491776->28495871] (req 28491776->28495871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.613348:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0745e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28491776->28495871] (req 28491776->28495871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.666361:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974275 is committed 00002000:00010000:2.0:1789760795.669405:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28491776 -> 28495872 00002000:00010000:2.0:1789760795.669415:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760794 -> 1789760795 00002000:00010000:2.0:1789760795.669417:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760794 -> 1789760795 00010000:00010000:2.0:1789760795.669431:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0745e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28491776->28495871] (req 28491776->28495871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.669443:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0745e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28491776->28495871] (req 28491776->28495871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.669451:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0745e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28491776->28495871] (req 28491776->28495871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.669463:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760795.669468:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0745e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28491776->28495871] (req 28491776->28495871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.669475:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0745e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28491776->28495871] (req 28491776->28495871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.669497:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0745e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28491776->28495871] (req 28491776->28495871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.669504:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e0745e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28491776->28495871] (req 28491776->28495871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.669625:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493876864:12345-192.168.203.60@tcp:4:dd.0 Request processed in 56373us (56602us total) trans 4294974275 rc 0/0 00000100:00100000:2.0:1789760795.669635:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6986 00000100:00100000:2.0:1789760795.669645:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805908927987 time 805954272429 state 0x2 ref 0 00000100:00100000:3.0:1789760795.677172:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760795.678220:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493876992 00000100:00100000:1.0:1789760795.678232:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493876992 00000100:00100000:1.0:1789760795.678297:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805908927987 time 805954272429 state 0x0 ref 1 00000100:00100000:1.0:1789760795.678314:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805954532845 time 805954272429 state 0x1 ref 1 00000100:00100000:1.0:1789760795.678352:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805954532845 time 806019388239 state 0x0 ref 1 00000100:00100000:1.0:1789760795.678366:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493876992:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760795.678417:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28495872, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe009 00010000:00010000:1.0:1789760795.678426:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07465 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.678443:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=806s v=(0 0) 00010000:00010000:1.0:1789760795.678447:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.678463:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07465 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28495872->28499967] (req 28495872->28499967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.678478:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07465 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28495872->28499967] (req 28495872->28499967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760795.749401:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28495872 -> 28499968 00010000:00010000:1.0:1789760795.749420:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07465 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28495872->28499967] (req 28495872->28499967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.749434:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07465 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28495872->28499967] (req 28495872->28499967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.749442:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07465 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28495872->28499967] (req 28495872->28499967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.749449:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760795.749454:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07465 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28495872->28499967] (req 28495872->28499967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.749462:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07465 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28495872->28499967] (req 28495872->28499967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.749482:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07465 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28495872->28499967] (req 28495872->28499967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.749490:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000744d38a0/0x3eb3dcc0b2e07465 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28495872->28499967] (req 28495872->28499967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.749592:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493876992:12345-192.168.203.60@tcp:4:dd.0 Request processed in 71224us (72424us total) trans 4294974276 rc 0/0 00000100:00100000:1.0:1789760795.749603:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6987 00000100:00100000:1.0:1789760795.749614:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805954532845 time 806019388239 state 0x2 ref 0 00000001:00080000:2.0:1789760795.749847:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974276 is committed 00000100:00100000:1.0:1789760795.754490:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760795.754584:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493877120 00000100:00100000:0.0:1789760795.754593:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493877120 00000100:00100000:0.0:1789760795.754649:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 805954532845 time 806019388239 state 0x0 ref 1 00000100:00100000:0.0:1789760795.754666:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806019648655 time 806019388239 state 0x1 ref 1 00000100:00100000:0.0:1789760795.754697:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806019648655 time 806095732269 state 0x0 ref 1 00000100:00100000:0.0:1789760795.754709:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493877120:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760795.754758:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28499968, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe009 00010000:00010000:0.0:1789760795.754767:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0746c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.754783:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=806s v=(0 0) 00010000:00010000:0.0:1789760795.754788:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.754795:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0746c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28499968->28504063] (req 28499968->28504063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.754809:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0746c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28499968->28504063] (req 28499968->28504063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760795.791762:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28499968 -> 28504064 00010000:00010000:3.0:1789760795.791784:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0746c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28499968->28504063] (req 28499968->28504063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.791806:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0746c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28499968->28504063] (req 28499968->28504063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.791820:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0746c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28499968->28504063] (req 28499968->28504063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.791828:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760795.791833:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0746c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28499968->28504063] (req 28499968->28504063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.791841:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0746c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28499968->28504063] (req 28499968->28504063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.791862:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0746c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28499968->28504063] (req 28499968->28504063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.791869:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0746c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28499968->28504063] (req 28499968->28504063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.792008:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493877120:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37294us (37522us total) trans 4294974277 rc 0/0 00000100:00100000:3.0:1789760795.792018:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6988 00000100:00100000:3.0:1789760795.792028:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806019648655 time 806095732269 state 0x2 ref 0 00000001:00080000:3.0:1789760795.794115:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974277 is committed 00000100:00100000:3.0:1789760795.796774:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760795.797257:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493877248 00000100:00100000:2.0:1789760795.797340:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493877248 00000100:00100000:2.0:1789760795.797398:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806019648655 time 806095732269 state 0x0 ref 1 00000100:00100000:2.0:1789760795.797414:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806095992685 time 806095732269 state 0x1 ref 1 00000100:00100000:2.0:1789760795.797444:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806095992685 time 806138479458 state 0x0 ref 1 00000100:00100000:2.0:1789760795.797456:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493877248:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760795.797505:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28504064, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe009 00010000:00010000:2.0:1789760795.797513:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e07473 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.797530:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=806s v=(0 0) 00010000:00010000:2.0:1789760795.797534:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.797540:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e07473 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28504064->28508159] (req 28504064->28508159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.797555:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e07473 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28504064->28508159] (req 28504064->28508159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.845136:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974278 is committed 00002000:00010000:1.0:1789760795.845478:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28504064 -> 28508160 00010000:00010000:1.0:1789760795.845494:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e07473 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28504064->28508159] (req 28504064->28508159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.845508:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e07473 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28504064->28508159] (req 28504064->28508159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.845518:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e07473 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28504064->28508159] (req 28504064->28508159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.845526:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760795.845532:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e07473 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28504064->28508159] (req 28504064->28508159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.845541:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e07473 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28504064->28508159] (req 28504064->28508159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.845562:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e07473 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28504064->28508159] (req 28504064->28508159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.845575:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e07473 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28504064->28508159] (req 28504064->28508159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.845883:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493877248:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48422us (49113us total) trans 4294974278 rc 0/0 00000100:00100000:1.0:1789760795.845895:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6989 00000100:00100000:1.0:1789760795.845908:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806095992685 time 806138479458 state 0x2 ref 0 00000100:00100000:1.0:1789760795.854463:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760795.854945:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493877376 00000100:00100000:2.0:1789760795.854955:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493877376 00000100:00100000:2.0:1789760795.855082:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806095992685 time 806138479458 state 0x0 ref 1 00000100:00100000:2.0:1789760795.855100:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806138739874 time 806138479458 state 0x1 ref 1 00000100:00100000:2.0:1789760795.855136:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806138739874 time 806196171541 state 0x0 ref 1 00000100:00100000:2.0:1789760795.855153:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493877376:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760795.855215:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28508160, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe009 00010000:00010000:2.0:1789760795.855224:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0747a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.855255:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=806s v=(0 0) 00010000:00010000:2.0:1789760795.855268:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.855275:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0747a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28508160->28512255] (req 28508160->28512255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.855289:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0747a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28508160->28512255] (req 28508160->28512255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.903422:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974279 is committed 00002000:00010000:3.0:1789760795.903527:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28508160 -> 28512256 00010000:00010000:3.0:1789760795.903544:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0747a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28508160->28512255] (req 28508160->28512255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.903556:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0747a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28508160->28512255] (req 28508160->28512255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.903565:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0747a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28508160->28512255] (req 28508160->28512255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.903572:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193de00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760795.903577:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0747a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28508160->28512255] (req 28508160->28512255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.903586:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0747a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28508160->28512255] (req 28508160->28512255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.903606:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0747a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28508160->28512255] (req 28508160->28512255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.903614:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047874222/0x3eb3dcc0b2e0747a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28508160->28512255] (req 28508160->28512255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.903800:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493877376:12345-192.168.203.60@tcp:4:dd.0 Request processed in 48645us (49339us total) trans 4294974279 rc 0/0 00000100:00100000:3.0:1789760795.903811:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6990 00000100:00100000:3.0:1789760795.903822:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806138739874 time 806196171541 state 0x2 ref 0 00000100:00100000:2.0:1789760795.910256:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760795.910532:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493877504 00000100:00100000:1.0:1789760795.910542:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493877504 00000100:00100000:1.0:1789760795.910603:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806138739874 time 806196171541 state 0x0 ref 1 00000100:00100000:1.0:1789760795.910622:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806196431957 time 806196171541 state 0x1 ref 1 00000100:00100000:1.0:1789760795.910692:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806196431957 time 806251726076 state 0x0 ref 1 00000100:00100000:1.0:1789760795.910705:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493877504:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760795.910762:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28512256, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe009 00010000:00010000:1.0:1789760795.910772:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07481 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.910788:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=806s v=(0 0) 00010000:00010000:1.0:1789760795.910792:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.910798:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07481 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28512256->28516351] (req 28512256->28516351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.910814:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07481 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28512256->28516351] (req 28512256->28516351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.952849:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974280 is committed 00002000:00010000:0.0:1789760795.953034:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28512256 -> 28516352 00010000:00010000:0.0:1789760795.953050:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07481 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28512256->28516351] (req 28512256->28516351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.953065:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07481 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28512256->28516351] (req 28512256->28516351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.953086:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07481 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28512256->28516351] (req 28512256->28516351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.953093:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760795.953099:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07481 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28512256->28516351] (req 28512256->28516351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.953109:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07481 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28512256->28516351] (req 28512256->28516351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.953131:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07481 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28512256->28516351] (req 28512256->28516351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.953138:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002142bbde/0x3eb3dcc0b2e07481 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28512256->28516351] (req 28512256->28516351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.953271:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493877504:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42562us (43019us total) trans 4294974280 rc 0/0 00000100:00100000:0.0:1789760795.953285:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6991 00000100:00100000:0.0:1789760795.953295:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806196431957 time 806251726076 state 0x2 ref 0 00000100:00100000:2.0:1789760795.957832:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760795.957926:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493877632 00000100:00100000:3.0:1789760795.957935:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493877632 00000100:00100000:3.0:1789760795.958034:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806196431957 time 806251726076 state 0x0 ref 1 00000100:00100000:3.0:1789760795.958050:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806251986492 time 806251726076 state 0x1 ref 1 00000100:00100000:3.0:1789760795.958089:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806251986492 time 806299124528 state 0x0 ref 1 00000100:00100000:3.0:1789760795.958101:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493877632:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760795.958151:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28516352, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe009 00010000:00010000:3.0:1789760795.958159:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07488 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.958174:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=806s v=(0 0) 00010000:00010000:3.0:1789760795.958179:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.958185:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07488 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28516352->28520447] (req 28516352->28520447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.958197:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07488 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28516352->28520447] (req 28516352->28520447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760795.986116:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28516352 -> 28520448 00010000:00010000:0.0:1789760795.986205:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07488 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28516352->28520447] (req 28516352->28520447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.986218:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07488 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28516352->28520447] (req 28516352->28520447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.986227:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07488 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28516352->28520447] (req 28516352->28520447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.986235:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760795.986240:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07488 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28516352->28520447] (req 28516352->28520447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.986249:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07488 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28516352->28520447] (req 28516352->28520447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.986269:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07488 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28516352->28520447] (req 28516352->28520447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.986281:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07488 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28516352->28520447] (req 28516352->28520447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.986378:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493877632:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28274us (28551us total) trans 4294974281 rc 0/0 00000100:00100000:0.0:1789760795.986388:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6992 00000100:00100000:0.0:1789760795.986399:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806251986492 time 806299124528 state 0x2 ref 0 00000001:00080000:1.0:1789760795.986991:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974281 is committed 00000100:00100000:1.0:1789760795.995389:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760795.995550:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493877760 00000100:00100000:2.0:1789760795.995560:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493877760 00000100:00100000:2.0:1789760795.995617:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806251986492 time 806299124528 state 0x0 ref 1 00000100:00100000:2.0:1789760795.995713:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806299384944 time 806299124528 state 0x1 ref 1 00000100:00100000:2.0:1789760795.995999:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806299384944 time 806336836532 state 0x0 ref 1 00000100:00100000:2.0:1789760795.996017:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493877760:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760795.996083:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28520448, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe009 00010000:00010000:2.0:1789760795.996092:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0748f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.996108:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=806s v=(0 0) 00010000:00010000:2.0:1789760795.996112:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.996120:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0748f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28520448->28524543] (req 28520448->28524543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.996133:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0748f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28520448->28524543] (req 28520448->28524543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.038532:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974282 is committed 00002000:00010000:1.0:1789760796.038542:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28520448 -> 28524544 00010000:00010000:1.0:1789760796.038559:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0748f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28520448->28524543] (req 28520448->28524543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.038573:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0748f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28520448->28524543] (req 28520448->28524543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.038582:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0748f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28520448->28524543] (req 28520448->28524543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.038589:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760796.038594:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0748f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28520448->28524543] (req 28520448->28524543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.038603:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0748f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28520448->28524543] (req 28520448->28524543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.038623:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0748f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28520448->28524543] (req 28520448->28524543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.038635:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f95cbfec/0x3eb3dcc0b2e0748f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28520448->28524543] (req 28520448->28524543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.038759:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d97100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493877760:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42742us (43376us total) trans 4294974282 rc 0/0 00000100:00100000:1.0:1789760796.038770:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6993 00000100:00100000:1.0:1789760796.038780:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806299384944 time 806336836532 state 0x2 ref 0 00000100:00100000:3.0:1789760796.044880:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760796.047229:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493877888 00000100:00100000:2.0:1789760796.047246:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493877888 00000100:00100000:2.0:1789760796.047312:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806299384944 time 806336836532 state 0x0 ref 1 00000100:00100000:2.0:1789760796.047328:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806337096948 time 806336836532 state 0x1 ref 1 00000100:00100000:2.0:1789760796.047367:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806337096948 time 806388402740 state 0x0 ref 1 00000100:00100000:2.0:1789760796.047381:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493877888:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760796.047432:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28524544, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe009 00010000:00010000:2.0:1789760796.047441:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e07496 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.047457:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=806s v=(0 0) 00010000:00010000:2.0:1789760796.047461:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.047467:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e07496 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28524544->28528639] (req 28524544->28528639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.047481:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e07496 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28524544->28528639] (req 28524544->28528639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.079242:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974283 is committed 00002000:00010000:3.0:1789760796.079360:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28524544 -> 28528640 00010000:00010000:3.0:1789760796.079375:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e07496 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28524544->28528639] (req 28524544->28528639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.079391:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e07496 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28524544->28528639] (req 28524544->28528639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.079399:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e07496 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28524544->28528639] (req 28524544->28528639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.079406:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760796.079411:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e07496 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28524544->28528639] (req 28524544->28528639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.079419:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e07496 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28524544->28528639] (req 28524544->28528639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.079438:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e07496 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28524544->28528639] (req 28524544->28528639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.079446:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e07496 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28524544->28528639] (req 28524544->28528639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.079540:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493877888:12345-192.168.203.60@tcp:4:dd.0 Request processed in 32156us (34665us total) trans 4294974283 rc 0/0 00000100:00100000:3.0:1789760796.079551:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6994 00000100:00100000:3.0:1789760796.079560:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806337096948 time 806388402740 state 0x2 ref 0 00000100:00100000:2.0:1789760796.085922:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760796.085992:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493878016 00000100:00100000:0.0:1789760796.086002:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493878016 00000100:00100000:0.0:1789760796.086059:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806337096948 time 806388402740 state 0x0 ref 1 00000100:00100000:0.0:1789760796.086087:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806388663156 time 806388402740 state 0x1 ref 1 00000100:00100000:0.0:1789760796.086122:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806388663156 time 806427157526 state 0x0 ref 1 00000100:00100000:0.0:1789760796.086135:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493878016:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760796.086184:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28528640, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe009 00010000:00010000:0.0:1789760796.086193:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0749d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.086208:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=806s v=(0 0) 00010000:00010000:0.0:1789760796.086213:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.086219:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0749d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28528640->28532735] (req 28528640->28532735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.086233:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0749d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28528640->28532735] (req 28528640->28532735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.123516:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974284 is committed 00002000:00010000:3.0:1789760796.123736:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28528640 -> 28532736 00010000:00010000:3.0:1789760796.123751:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0749d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28528640->28532735] (req 28528640->28532735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.123763:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0749d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28528640->28532735] (req 28528640->28532735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.123772:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0749d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28528640->28532735] (req 28528640->28532735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.123779:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afbc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760796.123783:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0749d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28528640->28532735] (req 28528640->28532735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.123792:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0749d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28528640->28532735] (req 28528640->28532735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.123811:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0749d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28528640->28532735] (req 28528640->28532735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.123819:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de6b4aa0/0x3eb3dcc0b2e0749d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28528640->28532735] (req 28528640->28532735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.123918:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493878016:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37779us (38001us total) trans 4294974284 rc 0/0 00000100:00100000:3.0:1789760796.123932:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6995 00000100:00100000:3.0:1789760796.123943:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806388663156 time 806427157526 state 0x2 ref 0 00000100:00100000:1.0:1789760796.128773:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760796.129223:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493878144 00000100:00100000:0.0:1789760796.129233:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493878144 00000100:00100000:0.0:1789760796.129406:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806388663156 time 806427157526 state 0x0 ref 1 00000100:00100000:0.0:1789760796.129425:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806427417942 time 806427157526 state 0x1 ref 1 00000100:00100000:0.0:1789760796.129533:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806427417942 time 806470566797 state 0x0 ref 1 00000100:00100000:0.0:1789760796.129561:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493878144:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760796.129622:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28532736, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe009 00010000:00010000:0.0:1789760796.129631:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e074a4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.129648:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=806s v=(0 0) 00010000:00010000:0.0:1789760796.129653:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.129660:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e074a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28532736->28536831] (req 28532736->28536831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.129674:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e074a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28532736->28536831] (req 28532736->28536831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.177082:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974285 is committed 00002000:00010000:3.0:1789760796.177190:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28532736 -> 28536832 00010000:00010000:3.0:1789760796.177207:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e074a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28532736->28536831] (req 28532736->28536831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.177220:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e074a4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28532736->28536831] (req 28532736->28536831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.177230:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e074a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28532736->28536831] (req 28532736->28536831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.177238:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760796.177243:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e074a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28532736->28536831] (req 28532736->28536831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.177253:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e074a4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28532736->28536831] (req 28532736->28536831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.177274:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e074a4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28532736->28536831] (req 28532736->28536831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.177281:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6aabd24/0x3eb3dcc0b2e074a4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28532736->28536831] (req 28532736->28536831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.177501:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493878144:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47952us (48733us total) trans 4294974285 rc 0/0 00000100:00100000:3.0:1789760796.177512:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6996 00000100:00100000:3.0:1789760796.177523:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806427417942 time 806470566797 state 0x2 ref 0 00000100:00100000:2.0:1789760796.181953:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760796.182079:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493878272 00000100:00100000:1.0:1789760796.182090:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493878272 00000100:00100000:1.0:1789760796.182203:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806427417942 time 806470566797 state 0x0 ref 1 00000100:00100000:1.0:1789760796.182218:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806470827213 time 806470566797 state 0x1 ref 1 00000100:00100000:1.0:1789760796.182239:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806470827213 time 806523275334 state 0x0 ref 1 00000100:00100000:1.0:1789760796.182251:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493878272:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760796.182364:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28536832, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe009 00010000:00010000:1.0:1789760796.182375:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e074ab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.182391:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=806s v=(0 0) 00010000:00010000:1.0:1789760796.182394:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.182400:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e074ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28536832->28540927] (req 28536832->28540927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.182415:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e074ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28536832->28540927] (req 28536832->28540927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.223078:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760796.234316:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974286 is committed 00002000:00010000:3.0:1789760796.234437:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28536832 -> 28540928 00010000:00010000:3.0:1789760796.234452:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e074ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28536832->28540927] (req 28536832->28540927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.234465:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e074ab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28536832->28540927] (req 28536832->28540927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.234474:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e074ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28536832->28540927] (req 28536832->28540927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.234480:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760796.234485:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e074ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28536832->28540927] (req 28536832->28540927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.234493:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e074ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28536832->28540927] (req 28536832->28540927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.234513:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e074ab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28536832->28540927] (req 28536832->28540927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.234520:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000bb3a7ca/0x3eb3dcc0b2e074ab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28536832->28540927] (req 28536832->28540927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.234609:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493878272:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52355us (52661us total) trans 4294974286 rc 0/0 00000100:00100000:3.0:1789760796.234623:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6997 00000100:00100000:3.0:1789760796.234632:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806470827213 time 806523275334 state 0x2 ref 0 00000100:00100000:2.0:1789760796.250299:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760796.250394:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493878400 00000100:00100000:3.0:1789760796.250405:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493878400 00000100:00100000:3.0:1789760796.250457:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806470827213 time 806523275334 state 0x0 ref 1 00000100:00100000:3.0:1789760796.250474:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806523535750 time 806523275334 state 0x1 ref 1 00000100:00100000:3.0:1789760796.250503:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806523535750 time 806591538253 state 0x0 ref 1 00000100:00100000:3.0:1789760796.250515:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493878400:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760796.250562:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28540928, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe009 00010000:00010000:3.0:1789760796.250571:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e074b2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.250586:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=806s v=(0 0) 00010000:00010000:3.0:1789760796.250590:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.250595:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e074b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28540928->28545023] (req 28540928->28545023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.250610:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e074b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28540928->28545023] (req 28540928->28545023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.301405:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974287 is committed 00002000:00010000:3.0:1789760796.301511:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28540928 -> 28545024 00010000:00010000:3.0:1789760796.301525:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e074b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28540928->28545023] (req 28540928->28545023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.301539:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e074b2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28540928->28545023] (req 28540928->28545023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.301557:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e074b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28540928->28545023] (req 28540928->28545023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.301565:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760796.301570:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e074b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28540928->28545023] (req 28540928->28545023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.301580:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e074b2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28540928->28545023] (req 28540928->28545023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.301601:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e074b2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28540928->28545023] (req 28540928->28545023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.301608:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e074b2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28540928->28545023] (req 28540928->28545023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.301697:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493878400:12345-192.168.203.60@tcp:4:dd.0 Request processed in 51180us (51407us total) trans 4294974287 rc 0/0 00000100:00100000:3.0:1789760796.301707:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6998 00000100:00100000:3.0:1789760796.301717:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806523535750 time 806591538253 state 0x2 ref 0 00000100:00100000:3.0:1789760796.304750:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760796.304904:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493878528 00000100:00100000:2.0:1789760796.304913:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493878528 00000100:00100000:2.0:1789760796.305003:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806523535750 time 806591538253 state 0x0 ref 1 00000100:00100000:2.0:1789760796.305019:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806591798669 time 806591538253 state 0x1 ref 1 00000100:00100000:2.0:1789760796.305039:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806591798669 time 806646075247 state 0x0 ref 1 00000100:00100000:2.0:1789760796.305051:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493878528:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760796.305107:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28545024, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe009 00010000:00010000:2.0:1789760796.305116:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e074b9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.305140:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=806s v=(0 0) 00010000:00010000:2.0:1789760796.305145:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.305152:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e074b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28545024->28549119] (req 28545024->28549119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.305165:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e074b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28545024->28549119] (req 28545024->28549119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.362344:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974288 is committed 00002000:00010000:2.0:1789760796.362495:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28545024 -> 28549120 00010000:00010000:2.0:1789760796.362511:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e074b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28545024->28549119] (req 28545024->28549119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.362523:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e074b9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28545024->28549119] (req 28545024->28549119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.362532:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e074b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28545024->28549119] (req 28545024->28549119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.362539:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ca00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760796.362543:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e074b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28545024->28549119] (req 28545024->28549119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.362552:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e074b9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28545024->28549119] (req 28545024->28549119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.362571:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e074b9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28545024->28549119] (req 28545024->28549119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.362579:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e92f93bf/0x3eb3dcc0b2e074b9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28545024->28549119] (req 28545024->28549119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.362763:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493878528:12345-192.168.203.60@tcp:4:dd.0 Request processed in 57709us (58016us total) trans 4294974288 rc 0/0 00000100:00100000:2.0:1789760796.362778:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 6999 00000100:00100000:2.0:1789760796.362788:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806591798669 time 806646075247 state 0x2 ref 0 00000100:00100000:2.0:1789760796.369825:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760796.369939:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493878656 00000100:00100000:3.0:1789760796.369949:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493878656 00000100:00100000:3.0:1789760796.370076:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806591798669 time 806646075247 state 0x0 ref 1 00000100:00100000:3.0:1789760796.370093:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806646335663 time 806646075247 state 0x1 ref 1 00000100:00100000:3.0:1789760796.370243:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806646335663 time 806711276955 state 0x0 ref 1 00000100:00100000:3.0:1789760796.370258:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493878656:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760796.370306:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28549120, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe009 00010000:00010000:3.0:1789760796.370314:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e074c0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.370332:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=806s v=(0 0) 00010000:00010000:3.0:1789760796.370336:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.370342:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e074c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28549120->28553215] (req 28549120->28553215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.370357:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e074c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28549120->28553215] (req 28549120->28553215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.406774:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974289 is committed 00002000:00010000:1.0:1789760796.406867:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28549120 -> 28553216 00010000:00010000:1.0:1789760796.406882:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e074c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28549120->28553215] (req 28549120->28553215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.406895:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e074c0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28549120->28553215] (req 28549120->28553215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.406904:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e074c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28549120->28553215] (req 28549120->28553215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.406911:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760796.406917:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e074c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28549120->28553215] (req 28549120->28553215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.406926:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e074c0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28549120->28553215] (req 28549120->28553215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.406945:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e074c0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28549120->28553215] (req 28549120->28553215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.406953:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006546524d/0x3eb3dcc0b2e074c0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28549120->28553215] (req 28549120->28553215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.407081:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493878656:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36811us (37249us total) trans 4294974289 rc 0/0 00000100:00100000:1.0:1789760796.407092:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7000 00000100:00100000:1.0:1789760796.407217:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806646335663 time 806711276955 state 0x2 ref 0 00000100:00100000:3.0:1789760796.411176:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760796.411210:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493878784 00000100:00100000:3.0:1789760796.411219:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493878784 00000100:00100000:3.0:1789760796.411268:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806646335663 time 806711276955 state 0x0 ref 1 00000100:00100000:3.0:1789760796.411283:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806711537371 time 806711276955 state 0x1 ref 1 00000100:00100000:3.0:1789760796.411305:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806711537371 time 806752340636 state 0x0 ref 1 00000100:00100000:3.0:1789760796.411317:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493878784:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760796.411363:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28553216, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe009 00010000:00010000:3.0:1789760796.411371:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074c7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.411385:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=806s v=(0 0) 00010000:00010000:3.0:1789760796.411390:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.411395:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28553216->28557311] (req 28553216->28557311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.411408:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28553216->28557311] (req 28553216->28557311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.446441:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974290 is committed 00002000:00010000:3.0:1789760796.446683:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28553216 -> 28557312 00010000:00010000:3.0:1789760796.446699:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28553216->28557311] (req 28553216->28557311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.446712:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074c7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28553216->28557311] (req 28553216->28557311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.446721:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28553216->28557311] (req 28553216->28557311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.446728:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760796.446733:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28553216->28557311] (req 28553216->28557311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.446743:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074c7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28553216->28557311] (req 28553216->28557311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.446763:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074c7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28553216->28557311] (req 28553216->28557311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.446772:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074c7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28553216->28557311] (req 28553216->28557311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.446908:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493878784:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35587us (35736us total) trans 4294974290 rc 0/0 00000100:00100000:3.0:1789760796.446918:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7001 00000100:00100000:3.0:1789760796.446928:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806711537371 time 806752340636 state 0x2 ref 0 00000100:00100000:3.0:1789760796.453705:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760796.454291:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493878912 00000100:00100000:2.0:1789760796.454301:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493878912 00000100:00100000:2.0:1789760796.454356:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806711537371 time 806752340636 state 0x0 ref 1 00000100:00100000:2.0:1789760796.454371:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806752601052 time 806752340636 state 0x1 ref 1 00000100:00100000:2.0:1789760796.454452:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806752601052 time 806795486692 state 0x0 ref 1 00000100:00100000:2.0:1789760796.454465:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493878912:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760796.454523:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28557312, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe009 00010000:00010000:2.0:1789760796.454533:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e074ce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.454820:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=806s v=(0 0) 00010000:00010000:2.0:1789760796.454827:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.454835:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e074ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28557312->28561407] (req 28557312->28561407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.454851:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e074ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28557312->28561407] (req 28557312->28561407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760796.492797:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28557312 -> 28561408 00010000:00010000:2.0:1789760796.492817:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e074ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28557312->28561407] (req 28557312->28561407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.492830:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e074ce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28557312->28561407] (req 28557312->28561407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.492838:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e074ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28557312->28561407] (req 28557312->28561407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.492846:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760796.492851:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e074ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28557312->28561407] (req 28557312->28561407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.492860:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e074ce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28557312->28561407] (req 28557312->28561407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.492877:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e074ce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28557312->28561407] (req 28557312->28561407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.492889:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e074ce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28557312->28561407] (req 28557312->28561407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.493011:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493878912:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38539us (39306us total) trans 4294974291 rc 0/0 00000100:00100000:2.0:1789760796.493023:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7002 00000100:00100000:2.0:1789760796.493032:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806752601052 time 806795486692 state 0x2 ref 0 00000001:00080000:3.0:1789760796.493115:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974291 is committed 00000100:00100000:2.0:1789760796.495489:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760796.495648:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493879040 00000100:00100000:0.0:1789760796.495656:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493879040 00000100:00100000:0.0:1789760796.495703:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806752601052 time 806795486692 state 0x0 ref 1 00000100:00100000:0.0:1789760796.495714:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806795747108 time 806795486692 state 0x1 ref 1 00000100:00100000:0.0:1789760796.495742:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806795747108 time 806836777356 state 0x0 ref 1 00000100:00100000:0.0:1789760796.495753:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493879040:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760796.495797:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28561408, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe009 00010000:00010000:0.0:1789760796.495805:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e074d5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.495821:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=806s v=(0 0) 00010000:00010000:0.0:1789760796.495826:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.495832:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e074d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28561408->28565503] (req 28561408->28565503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.495844:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e074d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28561408->28565503] (req 28561408->28565503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760796.523907:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28561408 -> 28565504 00010000:00010000:1.0:1789760796.523927:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e074d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28561408->28565503] (req 28561408->28565503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.523941:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e074d5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28561408->28565503] (req 28561408->28565503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.523950:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e074d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28561408->28565503] (req 28561408->28565503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.523958:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760796.523983:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e074d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28561408->28565503] (req 28561408->28565503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.523992:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e074d5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28561408->28565503] (req 28561408->28565503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.524012:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e074d5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28561408->28565503] (req 28561408->28565503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.524024:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd354a3f/0x3eb3dcc0b2e074d5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28561408->28565503] (req 28561408->28565503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.524278:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493879040:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28521us (28793us total) trans 4294974292 rc 0/0 00000100:00100000:1.0:1789760796.524289:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7003 00000100:00100000:1.0:1789760796.524299:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806795747108 time 806836777356 state 0x2 ref 0 00000001:00080000:3.0:1789760796.524805:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974292 is committed 00000100:00100000:1.0:1789760796.528524:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760796.528779:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493879168 00000100:00100000:3.0:1789760796.528788:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493879168 00000100:00100000:3.0:1789760796.528835:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806795747108 time 806836777356 state 0x0 ref 1 00000100:00100000:3.0:1789760796.528848:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806837037772 time 806836777356 state 0x1 ref 1 00000100:00100000:3.0:1789760796.528875:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806837037772 time 806869910521 state 0x0 ref 1 00000100:00100000:3.0:1789760796.528890:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493879168:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760796.528937:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28565504, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe009 00010000:00010000:3.0:1789760796.528946:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074dc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.528984:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=806s v=(0 0) 00010000:00010000:3.0:1789760796.528988:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.528994:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28565504->28569599] (req 28565504->28569599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.529006:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28565504->28569599] (req 28565504->28569599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760796.579680:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28565504 -> 28569600 00010000:00010000:3.0:1789760796.579712:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28565504->28569599] (req 28565504->28569599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.579725:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074dc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28565504->28569599] (req 28565504->28569599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.579734:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28565504->28569599] (req 28565504->28569599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.579740:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760796.579745:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28565504->28569599] (req 28565504->28569599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.579756:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074dc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28565504->28569599] (req 28565504->28569599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.579774:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074dc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28565504->28569599] (req 28565504->28569599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.579781:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074dc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28565504->28569599] (req 28565504->28569599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.579873:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493879168:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50980us (51352us total) trans 4294974293 rc 0/0 00000100:00100000:3.0:1789760796.579882:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7004 00000100:00100000:3.0:1789760796.579892:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806837037772 time 806869910521 state 0x2 ref 0 00000001:00080000:3.0:1789760796.579931:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974293 is committed 00000100:00100000:2.0:1789760796.587533:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760796.587624:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493879296 00000100:00100000:3.0:1789760796.587634:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493879296 00000100:00100000:3.0:1789760796.587692:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806837037772 time 806869910521 state 0x0 ref 1 00000100:00100000:3.0:1789760796.587707:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806870170937 time 806869910521 state 0x1 ref 1 00000100:00100000:3.0:1789760796.587739:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806870170937 time 806928774902 state 0x0 ref 1 00000100:00100000:3.0:1789760796.587751:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493879296:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760796.587805:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28569600, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xe009 00010000:00010000:3.0:1789760796.587815:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074e3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.587830:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=806s v=(0 0) 00010000:00010000:3.0:1789760796.587834:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.587840:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28569600->28573695] (req 28569600->28573695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.587852:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28569600->28573695] (req 28569600->28573695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.626083:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974294 is committed 00002000:00010000:3.0:1789760796.626249:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28569600 -> 28573696 00002000:00010000:3.0:1789760796.626255:0:6694:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760795 -> 1789760796 00002000:00010000:3.0:1789760796.626257:0:6694:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760795 -> 1789760796 00010000:00010000:3.0:1789760796.626269:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28569600->28573695] (req 28569600->28573695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.626282:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074e3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28569600->28573695] (req 28569600->28573695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.626291:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28569600->28573695] (req 28569600->28573695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.626298:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760796.626303:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28569600->28573695] (req 28569600->28573695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.626317:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074e3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28569600->28573695] (req 28569600->28573695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.626337:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074e3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28569600->28573695] (req 28569600->28573695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.626346:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074e3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28569600->28573695] (req 28569600->28573695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.626438:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2886680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493879296:12345-192.168.203.60@tcp:4:dd.0 Request processed in 38684us (38910us total) trans 4294974294 rc 0/0 00000100:00100000:3.0:1789760796.626449:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7005 00000100:00100000:3.0:1789760796.626458:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806870170937 time 806928774902 state 0x2 ref 0 00000100:00100000:2.0:1789760796.633038:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760796.633101:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493879424 00000100:00100000:3.0:1789760796.633111:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493879424 00000100:00100000:3.0:1789760796.633227:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806870170937 time 806928774902 state 0x0 ref 1 00000100:00100000:3.0:1789760796.633241:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806929035318 time 806928774902 state 0x1 ref 1 00000100:00100000:3.0:1789760796.633273:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806929035318 time 806974308619 state 0x0 ref 1 00000100:00100000:3.0:1789760796.633285:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2887b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493879424:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760796.633331:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28573696, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:3.0:1789760796.633340:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074ea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.633355:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=806s v=(0 0) 00010000:00010000:3.0:1789760796.633359:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.633366:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28573696->28577791] (req 28573696->28577791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.633380:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28573696->28577791] (req 28573696->28577791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760796.674459:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28573696 -> 28577792 00010000:00010000:0.0:1789760796.674480:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28573696->28577791] (req 28573696->28577791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.674493:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074ea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28573696->28577791] (req 28573696->28577791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.674504:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28573696->28577791] (req 28573696->28577791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.674512:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760796.674517:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28573696->28577791] (req 28573696->28577791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.674526:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074ea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28573696->28577791] (req 28573696->28577791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.674550:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074ea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28573696->28577791] (req 28573696->28577791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.674557:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076649533/0x3eb3dcc0b2e074ea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28573696->28577791] (req 28573696->28577791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.674651:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2887b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493879424:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41363us (41618us total) trans 4294974295 rc 0/0 00000100:00100000:0.0:1789760796.674661:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7006 00000100:00100000:0.0:1789760796.674671:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806929035318 time 806974308619 state 0x2 ref 0 00000001:00080000:1.0:1789760796.675705:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974295 is committed 00000100:00100000:1.0:1789760796.690687:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760796.690796:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493879552 00000100:00100000:2.0:1789760796.690805:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493879552 00000100:00100000:2.0:1789760796.690861:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806929035318 time 806974308619 state 0x0 ref 1 00000100:00100000:2.0:1789760796.690876:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806974569035 time 806974308619 state 0x1 ref 1 00000100:00100000:2.0:1789760796.690912:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806974569035 time 807031947628 state 0x0 ref 1 00000100:00100000:2.0:1789760796.690922:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493879552:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760796.691003:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28577792, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:2.0:1789760796.691012:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e074f1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.691027:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=807s v=(0 0) 00010000:00010000:2.0:1789760796.691032:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.691038:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e074f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28577792->28581887] (req 28577792->28581887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.691051:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e074f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28577792->28581887] (req 28577792->28581887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.738473:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974296 is committed 00002000:00010000:3.0:1789760796.738729:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28577792 -> 28581888 00010000:00010000:3.0:1789760796.738744:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e074f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28577792->28581887] (req 28577792->28581887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.738757:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e074f1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28577792->28581887] (req 28577792->28581887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.738766:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e074f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28577792->28581887] (req 28577792->28581887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.738773:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193da00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760796.738778:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e074f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28577792->28581887] (req 28577792->28581887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.738787:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e074f1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28577792->28581887] (req 28577792->28581887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.738806:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e074f1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28577792->28581887] (req 28577792->28581887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.738814:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000074c5baee/0x3eb3dcc0b2e074f1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28577792->28581887] (req 28577792->28581887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.738911:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493879552:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47984us (48228us total) trans 4294974296 rc 0/0 00000100:00100000:3.0:1789760796.738922:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7007 00000100:00100000:3.0:1789760796.738932:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806974569035 time 807031947628 state 0x2 ref 0 00000100:00100000:2.0:1789760796.747337:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760796.747646:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493879680 00000100:00100000:1.0:1789760796.747656:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493879680 00000100:00100000:1.0:1789760796.747713:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 806974569035 time 807031947628 state 0x0 ref 1 00000100:00100000:1.0:1789760796.747729:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807032208044 time 807031947628 state 0x1 ref 1 00000100:00100000:1.0:1789760796.747818:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807032208044 time 807088852585 state 0x0 ref 1 00000100:00100000:1.0:1789760796.747831:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2884380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493879680:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760796.747881:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28581888, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:1.0:1789760796.747890:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e074f8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.747910:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=807s v=(0 0) 00010000:00010000:1.0:1789760796.747915:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.747921:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e074f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28581888->28585983] (req 28581888->28585983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.747935:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e074f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28581888->28585983] (req 28581888->28585983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.779256:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974297 is committed 00002000:00010000:1.0:1789760796.779365:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28581888 -> 28585984 00010000:00010000:1.0:1789760796.779380:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e074f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28581888->28585983] (req 28581888->28585983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.779393:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e074f8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28581888->28585983] (req 28581888->28585983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.779402:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e074f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28581888->28585983] (req 28581888->28585983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.779410:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760796.779415:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e074f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28581888->28585983] (req 28581888->28585983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.779424:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e074f8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28581888->28585983] (req 28581888->28585983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.779452:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e074f8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28581888->28585983] (req 28581888->28585983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.779460:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e074f8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28581888->28585983] (req 28581888->28585983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.779556:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2884380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493879680:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31723us (32226us total) trans 4294974297 rc 0/0 00000100:00100000:1.0:1789760796.779566:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7008 00000100:00100000:1.0:1789760796.779576:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807032208044 time 807088852585 state 0x2 ref 0 00000100:00100000:2.0:1789760796.782854:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760796.783199:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493879808 00000100:00100000:3.0:1789760796.783209:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493879808 00000100:00100000:2.0:1789760796.783338:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807032208044 time 807088852585 state 0x0 ref 1 00000100:00100000:2.0:1789760796.783352:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807089113001 time 807088852585 state 0x1 ref 1 00000100:00100000:2.0:1789760796.783414:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807089113001 time 807124449925 state 0x0 ref 1 00000100:00100000:2.0:1789760796.783427:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493879808:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760796.783485:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28585984, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:2.0:1789760796.783494:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e074ff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.783510:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=807s v=(0 0) 00010000:00010000:2.0:1789760796.783513:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.783519:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e074ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28585984->28590079] (req 28585984->28590079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.783532:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e074ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28585984->28590079] (req 28585984->28590079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.823770:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974298 is committed 00002000:00010000:2.0:1789760796.823771:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28585984 -> 28590080 00010000:00010000:2.0:1789760796.823789:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e074ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28585984->28590079] (req 28585984->28590079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.823802:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e074ff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28585984->28590079] (req 28585984->28590079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.823811:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e074ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28585984->28590079] (req 28585984->28590079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.823880:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760796.823887:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e074ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28585984->28590079] (req 28585984->28590079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.823897:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e074ff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28585984->28590079] (req 28585984->28590079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.823916:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e074ff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28585984->28590079] (req 28585984->28590079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.823923:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e074ff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28585984->28590079] (req 28585984->28590079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.824050:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc2885c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493879808:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40621us (41199us total) trans 4294974298 rc 0/0 00000100:00100000:2.0:1789760796.824060:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7009 00000100:00100000:2.0:1789760796.824080:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807089113001 time 807124449925 state 0x2 ref 0 00000100:00100000:3.0:1789760796.827294:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760796.827892:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493879936 00000100:00100000:0.0:1789760796.827903:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493879936 00000100:00100000:0.0:1789760796.827995:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807089113001 time 807124449925 state 0x0 ref 1 00000100:00100000:0.0:1789760796.828011:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807124710341 time 807124449925 state 0x1 ref 1 00000100:00100000:0.0:1789760796.828032:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807124710341 time 807169068196 state 0x0 ref 1 00000100:00100000:0.0:1789760796.828044:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc48dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493879936:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760796.828102:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28590080, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:0.0:1789760796.828111:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07506 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.828133:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=807s v=(0 0) 00010000:00010000:0.0:1789760796.828138:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.828144:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07506 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28590080->28594175] (req 28590080->28594175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.828157:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07506 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28590080->28594175] (req 28590080->28594175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.883566:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974299 is committed 00002000:00010000:2.0:1789760796.883825:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28590080 -> 28594176 00010000:00010000:2.0:1789760796.883840:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07506 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28590080->28594175] (req 28590080->28594175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.883853:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07506 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28590080->28594175] (req 28590080->28594175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.883868:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07506 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28590080->28594175] (req 28590080->28594175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.883875:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760796.883885:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07506 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28590080->28594175] (req 28590080->28594175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.883895:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07506 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28590080->28594175] (req 28590080->28594175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.883915:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07506 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28590080->28594175] (req 28590080->28594175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.883923:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009032aa49/0x3eb3dcc0b2e07506 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28590080->28594175] (req 28590080->28594175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.884044:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc48dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493879936:12345-192.168.203.60@tcp:4:dd.0 Request processed in 55996us (56754us total) trans 4294974299 rc 0/0 00000100:00100000:2.0:1789760796.884054:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7010 00000100:00100000:2.0:1789760796.884072:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807124710341 time 807169068196 state 0x2 ref 0 00000100:00100000:3.0:1789760796.891189:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760796.891283:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493880064 00000100:00100000:2.0:1789760796.891294:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493880064 00000100:00100000:2.0:1789760796.891350:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807124710341 time 807169068196 state 0x0 ref 1 00000100:00100000:2.0:1789760796.891365:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807169328612 time 807169068196 state 0x1 ref 1 00000100:00100000:2.0:1789760796.891394:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807169328612 time 807232430680 state 0x0 ref 1 00000100:00100000:2.0:1789760796.891407:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493880064:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760796.891463:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28594176, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:2.0:1789760796.891472:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0750d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.891487:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=807s v=(0 0) 00010000:00010000:2.0:1789760796.891492:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.891499:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0750d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28594176->28598271] (req 28594176->28598271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.891513:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0750d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28594176->28598271] (req 28594176->28598271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.919193:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974300 is committed 00002000:00010000:2.0:1789760796.919353:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28594176 -> 28598272 00010000:00010000:2.0:1789760796.919367:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0750d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28594176->28598271] (req 28594176->28598271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.919379:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0750d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28594176->28598271] (req 28594176->28598271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.919388:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0750d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28594176->28598271] (req 28594176->28598271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.919395:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760796.919400:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0750d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28594176->28598271] (req 28594176->28598271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.919410:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0750d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28594176->28598271] (req 28594176->28598271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.919427:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0750d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28594176->28598271] (req 28594176->28598271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.919434:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfb28c/0x3eb3dcc0b2e0750d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28594176->28598271] (req 28594176->28598271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.919520:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493880064:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28111us (28338us total) trans 4294974300 rc 0/0 00000100:00100000:2.0:1789760796.919531:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7011 00000100:00100000:2.0:1789760796.919540:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807169328612 time 807232430680 state 0x2 ref 0 00000100:00100000:1.0:1789760796.924767:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760796.925130:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493880192 00000100:00100000:3.0:1789760796.925141:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493880192 00000100:00100000:3.0:1789760796.925194:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807169328612 time 807232430680 state 0x0 ref 1 00000100:00100000:3.0:1789760796.925208:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807232691096 time 807232430680 state 0x1 ref 1 00000100:00100000:3.0:1789760796.925239:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807232691096 time 807266275226 state 0x0 ref 1 00000100:00100000:3.0:1789760796.925252:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493880192:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760796.925297:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28598272, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:3.0:1789760796.925307:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07514 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.925321:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=807s v=(0 0) 00010000:00010000:3.0:1789760796.925326:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.925332:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07514 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28598272->28602367] (req 28598272->28602367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.925346:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07514 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28598272->28602367] (req 28598272->28602367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.965284:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974301 is committed 00002000:00010000:3.0:1789760796.967714:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28598272 -> 28602368 00010000:00010000:3.0:1789760796.967734:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07514 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28598272->28602367] (req 28598272->28602367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.967746:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07514 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28598272->28602367] (req 28598272->28602367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.967754:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07514 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28598272->28602367] (req 28598272->28602367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.967760:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760796.967765:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07514 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28598272->28602367] (req 28598272->28602367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.967773:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07514 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28598272->28602367] (req 28598272->28602367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.968288:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07514 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28598272->28602367] (req 28598272->28602367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.968298:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e07514 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28598272->28602367] (req 28598272->28602367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.968400:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d95180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493880192:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43146us (43639us total) trans 4294974301 rc 0/0 00000100:00100000:3.0:1789760796.968411:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7012 00000100:00100000:3.0:1789760796.968421:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807232691096 time 807266275226 state 0x2 ref 0 00000100:00100000:3.0:1789760796.972676:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760796.972874:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493880320 00000100:00100000:2.0:1789760796.972884:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493880320 00000100:00100000:2.0:1789760796.972938:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807232691096 time 807266275226 state 0x0 ref 1 00000100:00100000:2.0:1789760796.972953:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807266535642 time 807266275226 state 0x1 ref 1 00000100:00100000:2.0:1789760796.973021:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807266535642 time 807314055786 state 0x0 ref 1 00000100:00100000:2.0:1789760796.973033:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493880320:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760796.973092:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28602368, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:2.0:1789760796.973110:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0751b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.973238:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=807s v=(0 0) 00010000:00010000:2.0:1789760796.973243:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.973250:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0751b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28602368->28606463] (req 28602368->28606463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.973263:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0751b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28602368->28606463] (req 28602368->28606463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760797.014804:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28602368 -> 28606464 00010000:00010000:3.0:1789760797.014824:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0751b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28602368->28606463] (req 28602368->28606463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.014836:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0751b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28602368->28606463] (req 28602368->28606463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.014844:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0751b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28602368->28606463] (req 28602368->28606463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.014850:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193dc00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760797.014856:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0751b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28602368->28606463] (req 28602368->28606463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.014869:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0751b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28602368->28606463] (req 28602368->28606463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.014889:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0751b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28602368->28606463] (req 28602368->28606463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.014896:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067dcc03b/0x3eb3dcc0b2e0751b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28602368->28606463] (req 28602368->28606463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.015021:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb88700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2368:x1876699493880320:12345-192.168.203.60@tcp:4:dd.0 Request processed in 41985us (42350us total) trans 4294974302 rc 0/0 00000100:00100000:3.0:1789760797.015032:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7013 00000100:00100000:3.0:1789760797.015042:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807266535642 time 807314055786 state 0x2 ref 0 00000001:00080000:1.0:1789760797.017349:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974302 is committed 00000100:00100000:2.0:1789760797.018864:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760797.019083:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493880448 00000100:00100000:0.0:1789760797.019093:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493880448 00000100:00100000:0.0:1789760797.019165:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807266535642 time 807314055786 state 0x0 ref 1 00000100:00100000:0.0:1789760797.019179:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807314316202 time 807314055786 state 0x1 ref 1 00000100:00100000:0.0:1789760797.019207:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807314316202 time 807360243797 state 0x0 ref 1 00000100:00100000:0.0:1789760797.019219:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493880448:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760797.019268:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28606464, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:0.0:1789760797.019277:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e07522 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.019292:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=807s v=(0 0) 00010000:00010000:0.0:1789760797.019297:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.019303:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e07522 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28606464->28610559] (req 28606464->28610559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.019317:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e07522 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28606464->28610559] (req 28606464->28610559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760797.020234:0:5841:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789760797.047127:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28606464 -> 28610560 00010000:00010000:2.0:1789760797.047147:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e07522 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28606464->28610559] (req 28606464->28610559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.047158:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e07522 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28606464->28610559] (req 28606464->28610559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.047166:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e07522 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28606464->28610559] (req 28606464->28610559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.047172:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760797.047176:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e07522 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28606464->28610559] (req 28606464->28610559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.047184:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e07522 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28606464->28610559] (req 28606464->28610559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.047206:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e07522 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28606464->28610559] (req 28606464->28610559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.047213:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3882a82/0x3eb3dcc0b2e07522 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28606464->28610559] (req 28606464->28610559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.047258:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974303 is committed 00000100:00100000:2.0:1789760797.047338:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc1ec6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493880448:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28116us (28475us total) trans 4294974303 rc 0/0 00000100:00100000:2.0:1789760797.047350:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7014 00000100:00100000:2.0:1789760797.047360:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807314316202 time 807360243797 state 0x2 ref 0 00000100:00100000:1.0:1789760797.052325:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760797.052412:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493880576 00000100:00100000:0.0:1789760797.052422:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493880576 00000100:00100000:0.0:1789760797.052476:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807314316202 time 807360243797 state 0x0 ref 1 00000100:00100000:0.0:1789760797.052491:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807360504213 time 807360243797 state 0x1 ref 1 00000100:00100000:0.0:1789760797.052521:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807360504213 time 807393556478 state 0x0 ref 1 00000100:00100000:0.0:1789760797.052533:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493880576:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760797.052587:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28610560, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:0.0:1789760797.052597:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e07529 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.052612:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=807s v=(0 0) 00010000:00010000:0.0:1789760797.052616:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.052622:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e07529 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28610560->28614655] (req 28610560->28614655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.052637:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e07529 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28610560->28614655] (req 28610560->28614655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760797.085825:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28610560 -> 28614656 00010000:00010000:2.0:1789760797.085846:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e07529 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28610560->28614655] (req 28610560->28614655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.085860:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e07529 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28610560->28614655] (req 28610560->28614655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.085869:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e07529 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28610560->28614655] (req 28610560->28614655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.085876:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afaa00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760797.085881:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e07529 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28610560->28614655] (req 28610560->28614655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.085895:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e07529 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28610560->28614655] (req 28610560->28614655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.085915:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e07529 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28610560->28614655] (req 28610560->28614655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.085923:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d41a2a8/0x3eb3dcc0b2e07529 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28610560->28614655] (req 28610560->28614655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.086060:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d94e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493880576:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33525us (33739us total) trans 4294974304 rc 0/0 00000100:00100000:2.0:1789760797.086080:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7015 00000100:00100000:2.0:1789760797.086089:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807360504213 time 807393556478 state 0x2 ref 0 00000001:00080000:1.0:1789760797.087640:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974304 is committed 00000100:00100000:3.0:1789760797.090210:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760797.090438:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493880704 00000100:00100000:1.0:1789760797.090447:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493880704 00000100:00100000:1.0:1789760797.090502:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807360504213 time 807393556478 state 0x0 ref 1 00000100:00100000:1.0:1789760797.090517:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807393816894 time 807393556478 state 0x1 ref 1 00000100:00100000:1.0:1789760797.090554:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807393816894 time 807431578785 state 0x0 ref 1 00000100:00100000:1.0:1789760797.090568:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493880704:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760797.090614:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28614656, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:1.0:1789760797.090624:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07530 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.090639:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=807s v=(0 0) 00010000:00010000:1.0:1789760797.090644:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.090650:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07530 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28614656->28618751] (req 28614656->28618751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.090664:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07530 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28614656->28618751] (req 28614656->28618751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760797.122171:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28614656 -> 28618752 00010000:00010000:3.0:1789760797.122191:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07530 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28614656->28618751] (req 28614656->28618751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.122205:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07530 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28614656->28618751] (req 28614656->28618751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.122214:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07530 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28614656->28618751] (req 28614656->28618751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.122226:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760797.122232:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07530 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28614656->28618751] (req 28614656->28618751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.122241:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07530 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28614656->28618751] (req 28614656->28618751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.122259:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07530 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28614656->28618751] (req 28614656->28618751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.122266:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8285a44/0x3eb3dcc0b2e07530 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28614656->28618751] (req 28614656->28618751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.122362:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffb89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2367:x1876699493880704:12345-192.168.203.60@tcp:4:dd.0 Request processed in 31793us (32156us total) trans 4294974305 rc 0/0 00000100:00100000:3.0:1789760797.122373:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7016 00000100:00100000:3.0:1789760797.122383:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807393816894 time 807431578785 state 0x2 ref 0 00000001:00080000:0.0:1789760797.125444:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974305 is committed 00000100:00100000:3.0:1789760797.132284:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760797.132684:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493880832 00000100:00100000:1.0:1789760797.132694:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493880832 00000100:00100000:1.0:1789760797.132747:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807393816894 time 807431578785 state 0x0 ref 1 00000100:00100000:1.0:1789760797.132761:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807431839201 time 807431578785 state 0x1 ref 1 00000100:00100000:1.0:1789760797.132794:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807431839201 time 807473829442 state 0x0 ref 1 00000100:00100000:1.0:1789760797.132805:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493880832:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760797.132852:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28618752, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:1.0:1789760797.132861:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07537 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.132876:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=807s v=(0 0) 00010000:00010000:1.0:1789760797.132880:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.132887:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07537 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28618752->28622847] (req 28618752->28622847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.132905:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07537 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28618752->28622847] (req 28618752->28622847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760797.187232:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28618752 -> 28622848 00010000:00010000:2.0:1789760797.187253:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07537 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28618752->28622847] (req 28618752->28622847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.187266:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07537 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28618752->28622847] (req 28618752->28622847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.187275:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07537 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28618752->28622847] (req 28618752->28622847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.187282:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760797.187288:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07537 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28618752->28622847] (req 28618752->28622847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.187296:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07537 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28618752->28622847] (req 28618752->28622847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.187314:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07537 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28618752->28622847] (req 28618752->28622847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.187321:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07537 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28618752->28622847] (req 28618752->28622847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.187416:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493880832:12345-192.168.203.60@tcp:4:dd.0 Request processed in 54606us (55134us total) trans 4294974306 rc 0/0 00000100:00100000:2.0:1789760797.187427:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7017 00000100:00100000:2.0:1789760797.187437:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807431839201 time 807473829442 state 0x2 ref 0 00000001:00080000:0.0:1789760797.187683:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974306 is committed 00000100:00100000:2.0:1789760797.191461:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760797.191555:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493880960 00000100:00100000:0.0:1789760797.191565:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493880960 00000100:00100000:0.0:1789760797.191626:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807431839201 time 807473829442 state 0x0 ref 1 00000100:00100000:0.0:1789760797.191643:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807474089858 time 807473829442 state 0x1 ref 1 00000100:00100000:0.0:1789760797.191672:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807474089858 time 807532707815 state 0x0 ref 1 00000100:00100000:0.0:1789760797.191685:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ead80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493880960:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760797.191738:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28622848, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:0.0:1789760797.191747:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0753e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.191762:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=807s v=(0 0) 00010000:00010000:0.0:1789760797.191767:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.191774:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0753e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28622848->28626943] (req 28622848->28626943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.191798:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0753e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28622848->28626943] (req 28622848->28626943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.225356:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974307 is committed 00002000:00010000:0.0:1789760797.225416:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28622848 -> 28626944 00010000:00010000:0.0:1789760797.225431:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0753e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28622848->28626943] (req 28622848->28626943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.225443:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0753e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28622848->28626943] (req 28622848->28626943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.225452:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0753e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28622848->28626943] (req 28622848->28626943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.225459:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760797.225464:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0753e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28622848->28626943] (req 28622848->28626943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.225472:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0753e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28622848->28626943] (req 28622848->28626943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.225491:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0753e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28622848->28626943] (req 28622848->28626943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.225499:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0753e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28622848->28626943] (req 28622848->28626943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.225667:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ead80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493880960:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33979us (34211us total) trans 4294974307 rc 0/0 00000100:00100000:0.0:1789760797.225678:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7018 00000100:00100000:0.0:1789760797.225688:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807474089858 time 807532707815 state 0x2 ref 0 00000100:00100000:3.0:1789760797.229082:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760797.229409:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493881088 00000100:00100000:2.0:1789760797.229419:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493881088 00000100:00100000:2.0:1789760797.229477:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807474089858 time 807532707815 state 0x0 ref 1 00000100:00100000:2.0:1789760797.229492:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807532968231 time 807532707815 state 0x1 ref 1 00000100:00100000:2.0:1789760797.229525:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807532968231 time 807570561157 state 0x0 ref 1 00000100:00100000:2.0:1789760797.229537:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493881088:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760797.229587:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28626944, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:2.0:1789760797.229597:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07545 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.229613:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=807s v=(0 0) 00010000:00010000:2.0:1789760797.229618:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.229626:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07545 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28626944->28631039] (req 28626944->28631039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.229640:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07545 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28626944->28631039] (req 28626944->28631039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.243187:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760797.258561:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28626944 -> 28631040 00010000:00010000:0.0:1789760797.258582:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07545 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28626944->28631039] (req 28626944->28631039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.258595:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07545 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28626944->28631039] (req 28626944->28631039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.258605:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07545 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28626944->28631039] (req 28626944->28631039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.258612:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760797.258617:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07545 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28626944->28631039] (req 28626944->28631039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.258625:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07545 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28626944->28631039] (req 28626944->28631039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.258645:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07545 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28626944->28631039] (req 28626944->28631039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.258653:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54b2723/0x3eb3dcc0b2e07545 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28626944->28631039] (req 28626944->28631039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.258747:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493881088:12345-192.168.203.60@tcp:4:dd.0 Request processed in 29207us (29669us total) trans 4294974308 rc 0/0 00000100:00100000:0.0:1789760797.258757:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7019 00000100:00100000:0.0:1789760797.258766:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807532968231 time 807570561157 state 0x2 ref 0 00000001:00080000:3.0:1789760797.259435:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974308 is committed 00000100:00100000:3.0:1789760797.262107:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760797.262243:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493881216 00000100:00100000:1.0:1789760797.262253:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493881216 00000100:00100000:1.0:1789760797.262307:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807532968231 time 807570561157 state 0x0 ref 1 00000100:00100000:1.0:1789760797.262322:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807570821573 time 807570561157 state 0x1 ref 1 00000100:00100000:1.0:1789760797.262347:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807570821573 time 807603383361 state 0x0 ref 1 00000100:00100000:1.0:1789760797.262360:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620ffe87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493881216:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760797.262409:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28631040, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:1.0:1789760797.262417:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0754c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.262432:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=807s v=(0 0) 00010000:00010000:1.0:1789760797.262436:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.262442:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0754c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28631040->28635135] (req 28631040->28635135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.262455:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0754c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28631040->28635135] (req 28631040->28635135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.297763:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974309 is committed 00002000:00010000:1.0:1789760797.297873:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28631040 -> 28635136 00010000:00010000:1.0:1789760797.297888:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0754c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28631040->28635135] (req 28631040->28635135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.297902:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0754c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28631040->28635135] (req 28631040->28635135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.297911:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0754c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28631040->28635135] (req 28631040->28635135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.297919:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760797.297924:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0754c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28631040->28635135] (req 28631040->28635135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.298217:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0754c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28631040->28635135] (req 28631040->28635135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.298237:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0754c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28631040->28635135] (req 28631040->28635135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.298244:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e0754c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28631040->28635135] (req 28631040->28635135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.298327:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620ffe87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493881216:12345-192.168.203.60@tcp:4:dd.0 Request processed in 35965us (36224us total) trans 4294974309 rc 0/0 00000100:00100000:1.0:1789760797.298339:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7020 00000100:00100000:1.0:1789760797.298348:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807570821573 time 807603383361 state 0x2 ref 0 00000100:00100000:2.0:1789760797.305174:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760797.305255:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493881344 00000100:00100000:1.0:1789760797.305266:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493881344 00000100:00100000:1.0:1789760797.305320:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807570821573 time 807603383361 state 0x0 ref 1 00000100:00100000:1.0:1789760797.305336:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807603643777 time 807603383361 state 0x1 ref 1 00000100:00100000:1.0:1789760797.305364:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807603643777 time 807646399707 state 0x0 ref 1 00000100:00100000:1.0:1789760797.305376:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493881344:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760797.305423:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28635136, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:1.0:1789760797.305432:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07553 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.305448:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=807s v=(0 0) 00010000:00010000:1.0:1789760797.305453:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.305459:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07553 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28635136->28639231] (req 28635136->28639231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.305472:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07553 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28635136->28639231] (req 28635136->28639231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.352800:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974310 is committed 00002000:00010000:2.0:1789760797.352918:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28635136 -> 28639232 00010000:00010000:2.0:1789760797.352935:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07553 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28635136->28639231] (req 28635136->28639231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.352948:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07553 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28635136->28639231] (req 28635136->28639231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.352957:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07553 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28635136->28639231] (req 28635136->28639231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.352982:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760797.352988:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07553 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28635136->28639231] (req 28635136->28639231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.353003:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07553 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28635136->28639231] (req 28635136->28639231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.353024:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07553 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28635136->28639231] (req 28635136->28639231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.353031:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c806af86/0x3eb3dcc0b2e07553 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28635136->28639231] (req 28635136->28639231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.353207:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493881344:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47827us (48036us total) trans 4294974310 rc 0/0 00000100:00100000:2.0:1789760797.353218:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7021 00000100:00100000:2.0:1789760797.353228:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807603643777 time 807646399707 state 0x2 ref 0 00000100:00100000:3.0:1789760797.356883:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760797.357058:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493881472 00000100:00100000:0.0:1789760797.357075:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493881472 00000100:00100000:0.0:1789760797.357137:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807603643777 time 807646399707 state 0x0 ref 1 00000100:00100000:0.0:1789760797.357152:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807646660123 time 807646399707 state 0x1 ref 1 00000100:00100000:0.0:1789760797.357229:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807646660123 time 807698264723 state 0x0 ref 1 00000100:00100000:0.0:1789760797.357242:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493881472:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760797.357291:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28639232, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:0.0:1789760797.357300:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0755a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.357315:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=807s v=(0 0) 00010000:00010000:0.0:1789760797.357319:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.357325:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0755a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28639232->28643327] (req 28639232->28643327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.357338:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0755a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28639232->28643327] (req 28639232->28643327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.400347:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974311 is committed 00002000:00010000:2.0:1789760797.400682:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28639232 -> 28643328 00010000:00010000:2.0:1789760797.400697:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0755a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28639232->28643327] (req 28639232->28643327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.400709:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0755a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28639232->28643327] (req 28639232->28643327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.400717:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0755a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28639232->28643327] (req 28639232->28643327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.400725:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760797.400730:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0755a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28639232->28643327] (req 28639232->28643327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.400740:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0755a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28639232->28643327] (req 28639232->28643327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.400758:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0755a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28639232->28643327] (req 28639232->28643327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.400766:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a95d8878/0x3eb3dcc0b2e0755a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28639232->28643327] (req 28639232->28643327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.400864:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c7aa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493881472:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43620us (43985us total) trans 4294974311 rc 0/0 00000100:00100000:2.0:1789760797.400897:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7022 00000100:00100000:2.0:1789760797.401162:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807646660123 time 807698264723 state 0x2 ref 0 00000100:00100000:3.0:1789760797.405478:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760797.405577:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493881600 00000100:00100000:2.0:1789760797.405586:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493881600 00000100:00100000:2.0:1789760797.405638:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807646660123 time 807698264723 state 0x0 ref 1 00000100:00100000:2.0:1789760797.405693:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807698525139 time 807698264723 state 0x1 ref 1 00000100:00100000:2.0:1789760797.405724:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807698525139 time 807746760496 state 0x0 ref 1 00000100:00100000:2.0:1789760797.405736:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620c8794700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493881600:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760797.405781:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28643328, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:2.0:1789760797.405794:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e07561 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.405809:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=807s v=(0 0) 00010000:00010000:2.0:1789760797.405825:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.405832:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e07561 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28643328->28647423] (req 28643328->28647423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.405846:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e07561 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28643328->28647423] (req 28643328->28647423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760797.447675:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28643328 -> 28647424 00010000:00010000:2.0:1789760797.447694:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e07561 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28643328->28647423] (req 28643328->28647423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.448069:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e07561 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28643328->28647423] (req 28643328->28647423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.448080:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e07561 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28643328->28647423] (req 28643328->28647423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.448207:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193d200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760797.448212:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e07561 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28643328->28647423] (req 28643328->28647423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.448222:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e07561 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28643328->28647423] (req 28643328->28647423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.448247:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e07561 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28643328->28647423] (req 28643328->28647423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.448256:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b5317/0x3eb3dcc0b2e07561 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28643328->28647423] (req 28643328->28647423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.449644:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620c8794700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493881600:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43902us (44168us total) trans 4294974312 rc 0/0 00000100:00100000:2.0:1789760797.449657:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7023 00000100:00100000:2.0:1789760797.449666:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807698525139 time 807746760496 state 0x2 ref 0 00000001:00080000:2.0:1789760797.449718:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974312 is committed 00000100:00100000:1.0:1789760797.459535:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760797.459680:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493881728 00000100:00100000:3.0:1789760797.459690:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493881728 00000100:00100000:3.0:1789760797.459755:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807698525139 time 807746760496 state 0x0 ref 1 00000100:00100000:3.0:1789760797.459778:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807747020912 time 807746760496 state 0x1 ref 1 00000100:00100000:3.0:1789760797.459811:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807747020912 time 807800846776 state 0x0 ref 1 00000100:00100000:3.0:1789760797.459824:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d96d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493881728:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760797.459872:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28647424, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:3.0:1789760797.459882:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07568 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.460400:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=807s v=(0 0) 00010000:00010000:3.0:1789760797.460405:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.460411:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07568 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28647424->28651519] (req 28647424->28651519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.460426:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07568 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28647424->28651519] (req 28647424->28651519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760797.520830:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28647424 -> 28651520 00010000:00010000:1.0:1789760797.520850:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07568 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28647424->28651519] (req 28647424->28651519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.520867:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07568 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28647424->28651519] (req 28647424->28651519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.520876:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07568 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28647424->28651519] (req 28647424->28651519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.520883:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdb600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760797.520887:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07568 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28647424->28651519] (req 28647424->28651519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.520896:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07568 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28647424->28651519] (req 28647424->28651519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.520914:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07568 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28647424->28651519] (req 28647424->28651519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.520922:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd4d0728/0x3eb3dcc0b2e07568 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28647424->28651519] (req 28647424->28651519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.521173:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4d96d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493881728:12345-192.168.203.60@tcp:4:dd.0 Request processed in 61346us (61643us total) trans 4294974313 rc 0/0 00000100:00100000:1.0:1789760797.521185:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7024 00000100:00100000:1.0:1789760797.521195:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807747020912 time 807800846776 state 0x2 ref 0 00000001:00080000:0.0:1789760797.521464:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974313 is committed 00000100:00100000:3.0:1789760797.526712:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760797.530532:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493881856 00000100:00100000:2.0:1789760797.530547:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493881856 00000100:00100000:2.0:1789760797.530608:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807747020912 time 807800846776 state 0x0 ref 1 00000100:00100000:2.0:1789760797.530624:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807801107192 time 807800846776 state 0x1 ref 1 00000100:00100000:2.0:1789760797.530694:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807801107192 time 807871728915 state 0x0 ref 1 00000100:00100000:2.0:1789760797.530709:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb450000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493881856:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760797.530758:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28651520, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:2.0:1789760797.530768:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0756f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.530802:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=807s v=(0 0) 00010000:00010000:2.0:1789760797.530807:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.530815:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0756f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28651520->28655615] (req 28651520->28655615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.530831:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0756f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28651520->28655615] (req 28651520->28655615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760797.573854:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28651520 -> 28655616 00010000:00010000:3.0:1789760797.573873:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0756f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28651520->28655615] (req 28651520->28655615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.573894:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0756f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28651520->28655615] (req 28651520->28655615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.573903:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0756f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28651520->28655615] (req 28651520->28655615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.573910:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193c400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760797.573915:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0756f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28651520->28655615] (req 28651520->28655615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.573924:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0756f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28651520->28655615] (req 28651520->28655615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.573945:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0756f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28651520->28655615] (req 28651520->28655615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.573952:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006278c137/0x3eb3dcc0b2e0756f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28651520->28655615] (req 28651520->28655615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.574074:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb450000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493881856:12345-192.168.203.60@tcp:4:dd.0 Request processed in 43352us (47356us total) trans 4294974314 rc 0/0 00000100:00100000:3.0:1789760797.574085:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7025 00000100:00100000:3.0:1789760797.574095:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807801107192 time 807871728915 state 0x2 ref 0 00000001:00080000:0.0:1789760797.574226:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974314 is committed 00000100:00100000:2.0:1789760797.578871:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760797.579010:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493881984 00000100:00100000:0.0:1789760797.579020:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493881984 00000100:00100000:0.0:1789760797.579085:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807801107192 time 807871728915 state 0x0 ref 1 00000100:00100000:0.0:1789760797.579100:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807871989331 time 807871728915 state 0x1 ref 1 00000100:00100000:0.0:1789760797.579132:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807871989331 time 807920168001 state 0x0 ref 1 00000100:00100000:0.0:1789760797.579144:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493881984:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760797.579191:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28655616, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xe009 00010000:00010000:0.0:1789760797.579200:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07576 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.579216:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=807s v=(0 0) 00010000:00010000:0.0:1789760797.579221:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.579227:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07576 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28655616->28659711] (req 28655616->28659711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.579240:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07576 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28655616->28659711] (req 28655616->28659711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.628927:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974315 is committed 00002000:00010000:0.0:1789760797.629055:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28655616 -> 28659712 00002000:00010000:0.0:1789760797.629068:0:6692:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789760796 -> 1789760797 00002000:00010000:0.0:1789760797.629071:0:6692:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789760796 -> 1789760797 00010000:00010000:0.0:1789760797.629083:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07576 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28655616->28659711] (req 28655616->28659711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.629096:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07576 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28655616->28659711] (req 28655616->28659711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.629104:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07576 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28655616->28659711] (req 28655616->28659711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.629111:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760797.629116:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07576 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28655616->28659711] (req 28655616->28659711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.629133:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07576 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28655616->28659711] (req 28655616->28659711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.629151:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07576 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28655616->28659711] (req 28655616->28659711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.629159:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07576 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28655616->28659711] (req 28655616->28659711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.629614:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493881984:12345-192.168.203.60@tcp:4:dd.0 Request processed in 50466us (50749us total) trans 4294974315 rc 0/0 00000100:00100000:0.0:1789760797.629625:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7026 00000100:00100000:0.0:1789760797.629636:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807871989331 time 807920168001 state 0x2 ref 0 00000100:00100000:2.0:1789760797.634102:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760797.634343:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493882112 00000100:00100000:0.0:1789760797.634353:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493882112 00000100:00100000:0.0:1789760797.634402:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807871989331 time 807920168001 state 0x0 ref 1 00000100:00100000:0.0:1789760797.634417:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807920428417 time 807920168001 state 0x1 ref 1 00000100:00100000:0.0:1789760797.634444:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807920428417 time 807975479961 state 0x0 ref 1 00000100:00100000:0.0:1789760797.634455:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493882112:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760797.634498:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28659712, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:0.0:1789760797.634507:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0757d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.634522:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=807s v=(0 0) 00010000:00010000:0.0:1789760797.634526:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.634532:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0757d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28659712->28663807] (req 28659712->28663807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.634545:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0757d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28659712->28663807] (req 28659712->28663807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.667486:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974316 is committed 00002000:00010000:1.0:1789760797.667720:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28659712 -> 28663808 00010000:00010000:1.0:1789760797.667735:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0757d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28659712->28663807] (req 28659712->28663807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.667747:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0757d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28659712->28663807] (req 28659712->28663807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.667756:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0757d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28659712->28663807] (req 28659712->28663807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.667763:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760797.667768:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0757d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28659712->28663807] (req 28659712->28663807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.667776:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0757d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28659712->28663807] (req 28659712->28663807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.667797:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0757d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28659712->28663807] (req 28659712->28663807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.667804:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004490adba/0x3eb3dcc0b2e0757d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28659712->28663807] (req 28659712->28663807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.667979:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493882112:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33499us (33860us total) trans 4294974316 rc 0/0 00000100:00100000:1.0:1789760797.667990:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7027 00000100:00100000:1.0:1789760797.668000:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807920428417 time 807975479961 state 0x2 ref 0 00000100:00100000:3.0:1789760797.681844:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760797.682017:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493882240 00000100:00100000:1.0:1789760797.682028:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493882240 00000100:00100000:1.0:1789760797.682092:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807920428417 time 807975479961 state 0x0 ref 1 00000100:00100000:1.0:1789760797.682107:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807975740377 time 807975479961 state 0x1 ref 1 00000100:00100000:1.0:1789760797.682139:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807975740377 time 808023174221 state 0x0 ref 1 00000100:00100000:1.0:1789760797.682151:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb453480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493882240:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760797.682198:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28663808, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:1.0:1789760797.682207:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e07584 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.682223:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=808s v=(0 0) 00010000:00010000:1.0:1789760797.682228:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.682234:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e07584 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28663808->28667903] (req 28663808->28667903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.682248:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e07584 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28663808->28667903] (req 28663808->28667903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760797.717954:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28663808 -> 28667904 00010000:00010000:1.0:1789760797.718004:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e07584 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28663808->28667903] (req 28663808->28667903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.718017:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e07584 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28663808->28667903] (req 28663808->28667903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.718027:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e07584 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28663808->28667903] (req 28663808->28667903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.718034:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1494c00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760797.718039:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e07584 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28663808->28667903] (req 28663808->28667903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.718048:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e07584 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28663808->28667903] (req 28663808->28667903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.718090:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e07584 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28663808->28667903] (req 28663808->28667903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.718098:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005c70573/0x3eb3dcc0b2e07584 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28663808->28667903] (req 28663808->28667903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.718189:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974317 is committed 00000100:00100000:1.0:1789760797.718328:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb453480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493882240:12345-192.168.203.60@tcp:4:dd.0 Request processed in 36173us (36487us total) trans 4294974317 rc 0/0 00000100:00100000:1.0:1789760797.718340:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7028 00000100:00100000:1.0:1789760797.718350:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807975740377 time 808023174221 state 0x2 ref 0 00000100:00100000:2.0:1789760797.723795:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760797.723911:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493882368 00000100:00100000:3.0:1789760797.723921:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493882368 00000100:00100000:3.0:1789760797.723989:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 807975740377 time 808023174221 state 0x0 ref 1 00000100:00100000:3.0:1789760797.724004:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808023434637 time 808023174221 state 0x1 ref 1 00000100:00100000:3.0:1789760797.724035:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808023434637 time 808065070828 state 0x0 ref 1 00000100:00100000:3.0:1789760797.724047:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493882368:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760797.724424:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28667904, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:3.0:1789760797.724435:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0758b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.724451:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=808s v=(0 0) 00010000:00010000:3.0:1789760797.724456:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.724461:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0758b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28667904->28671999] (req 28667904->28671999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.724476:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0758b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28667904->28671999] (req 28667904->28671999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.775699:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974318 is committed 00002000:00010000:1.0:1789760797.775821:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28667904 -> 28672000 00010000:00010000:1.0:1789760797.775838:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0758b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28667904->28671999] (req 28667904->28671999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.775857:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0758b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28667904->28671999] (req 28667904->28671999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.775866:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0758b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28667904->28671999] (req 28667904->28671999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.775874:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fdba00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760797.775879:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0758b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28667904->28671999] (req 28667904->28671999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.775888:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0758b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28667904->28671999] (req 28667904->28671999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.775909:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0758b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28667904->28671999] (req 28667904->28671999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.775917:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaf5a24/0x3eb3dcc0b2e0758b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28667904->28671999] (req 28667904->28671999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.776074:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493882368:12345-192.168.203.60@tcp:4:dd.0 Request processed in 52015us (52275us total) trans 4294974318 rc 0/0 00000100:00100000:1.0:1789760797.776482:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7029 00000100:00100000:1.0:1789760797.776492:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808023434637 time 808065070828 state 0x2 ref 0 00000100:00100000:2.0:1789760797.783166:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760797.783301:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493882496 00000100:00100000:0.0:1789760797.783312:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493882496 00000100:00100000:0.0:1789760797.783364:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808023434637 time 808065070828 state 0x0 ref 1 00000100:00100000:0.0:1789760797.783379:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808065331244 time 808065070828 state 0x1 ref 1 00000100:00100000:0.0:1789760797.783415:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808065331244 time 808124450901 state 0x0 ref 1 00000100:00100000:0.0:1789760797.783427:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493882496:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760797.783477:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28672000, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:0.0:1789760797.783486:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07592 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.783501:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=808s v=(0 0) 00010000:00010000:0.0:1789760797.783505:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.783511:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07592 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28672000->28676095] (req 28672000->28676095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.783525:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07592 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28672000->28676095] (req 28672000->28676095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.824736:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974319 is committed 00002000:00010000:0.0:1789760797.825170:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28672000 -> 28676096 00010000:00010000:0.0:1789760797.825187:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07592 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28672000->28676095] (req 28672000->28676095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.825202:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07592 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28672000->28676095] (req 28672000->28676095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.825212:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07592 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28672000->28676095] (req 28672000->28676095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.825219:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760797.825224:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07592 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28672000->28676095] (req 28672000->28676095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.825233:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07592 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28672000->28676095] (req 28672000->28676095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.825251:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07592 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28672000->28676095] (req 28672000->28676095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.825258:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e07592 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28672000->28676095] (req 28672000->28676095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.825487:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493882496:12345-192.168.203.60@tcp:4:dd.0 Request processed in 42056us (42334us total) trans 4294974319 rc 0/0 00000100:00100000:0.0:1789760797.825497:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7030 00000100:00100000:0.0:1789760797.825505:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808065331244 time 808124450901 state 0x2 ref 0 00000100:00100000:3.0:1789760797.828852:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760797.829029:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493882624 00000100:00100000:0.0:1789760797.829040:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493882624 00000100:00100000:0.0:1789760797.829101:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808065331244 time 808124450901 state 0x0 ref 1 00000100:00100000:0.0:1789760797.829117:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808124711317 time 808124450901 state 0x1 ref 1 00000100:00100000:0.0:1789760797.829138:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808124711317 time 808170174685 state 0x0 ref 1 00000100:00100000:0.0:1789760797.829152:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb450e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493882624:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760797.829201:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28676096, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:0.0:1789760797.829210:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e07599 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.829226:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=808s v=(0 0) 00010000:00010000:0.0:1789760797.829230:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.829237:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e07599 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28676096->28680191] (req 28676096->28680191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.829250:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e07599 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28676096->28680191] (req 28676096->28680191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760797.887700:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28676096 -> 28680192 00010000:00010000:0.0:1789760797.887720:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e07599 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28676096->28680191] (req 28676096->28680191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.887745:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e07599 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28676096->28680191] (req 28676096->28680191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.887755:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e07599 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28676096->28680191] (req 28676096->28680191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.887763:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760797.887769:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e07599 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28676096->28680191] (req 28676096->28680191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.887778:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e07599 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28676096->28680191] (req 28676096->28680191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.887806:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e07599 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28676096->28680191] (req 28676096->28680191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.887813:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e07599 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28676096->28680191] (req 28676096->28680191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.887906:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb450e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493882624:12345-192.168.203.60@tcp:4:dd.0 Request processed in 58752us (59059us total) trans 4294974320 rc 0/0 00000100:00100000:0.0:1789760797.887916:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7031 00000100:00100000:0.0:1789760797.887925:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808124711317 time 808170174685 state 0x2 ref 0 00000001:00080000:3.0:1789760797.889899:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974320 is committed 00000100:00100000:2.0:1789760797.892052:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760797.892480:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493882752 00000100:00100000:1.0:1789760797.892490:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493882752 00000100:00100000:1.0:1789760797.892563:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808124711317 time 808170174685 state 0x0 ref 1 00000100:00100000:1.0:1789760797.892578:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808170435101 time 808170174685 state 0x1 ref 1 00000100:00100000:1.0:1789760797.892606:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808170435101 time 808233641639 state 0x0 ref 1 00000100:00100000:1.0:1789760797.892617:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493882752:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760797.892665:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28680192, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:1.0:1789760797.892675:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e075a0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.892690:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=808s v=(0 0) 00010000:00010000:1.0:1789760797.892694:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.892700:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e075a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28680192->28684287] (req 28680192->28684287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.892714:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e075a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28680192->28684287] (req 28680192->28684287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760797.945807:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28680192 -> 28684288 00010000:00010000:1.0:1789760797.945827:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e075a0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28680192->28684287] (req 28680192->28684287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.945842:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e075a0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28680192->28684287] (req 28680192->28684287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.945856:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e075a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28680192->28684287] (req 28680192->28684287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.945863:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495e00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760797.945867:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e075a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28680192->28684287] (req 28680192->28684287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.945877:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e075a0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28680192->28684287] (req 28680192->28684287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.945896:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e075a0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28680192->28684287] (req 28680192->28684287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.945904:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b0235ca/0x3eb3dcc0b2e075a0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28680192->28684287] (req 28680192->28684287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.946034:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493882752:12345-192.168.203.60@tcp:4:dd.0 Request processed in 53414us (53986us total) trans 4294974321 rc 0/0 00000100:00100000:1.0:1789760797.946046:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7032 00000100:00100000:1.0:1789760797.946067:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808170435101 time 808233641639 state 0x2 ref 0 00000001:00080000:3.0:1789760797.946373:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974321 is committed 00000100:00100000:1.0:1789760797.950998:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760797.951094:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493882880 00000100:00100000:3.0:1789760797.951105:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493882880 00000100:00100000:3.0:1789760797.951172:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808170435101 time 808233641639 state 0x0 ref 1 00000100:00100000:3.0:1789760797.951187:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808233902055 time 808233641639 state 0x1 ref 1 00000100:00100000:3.0:1789760797.951220:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808233902055 time 808292256495 state 0x0 ref 1 00000100:00100000:3.0:1789760797.951232:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4529f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493882880:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760797.951279:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28684288, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:3.0:1789760797.951289:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e075a7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.951304:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=808s v=(0 0) 00010000:00010000:3.0:1789760797.951310:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.951316:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e075a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28684288->28688383] (req 28684288->28688383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.951331:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e075a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28684288->28688383] (req 28684288->28688383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.998454:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974322 is committed 00002000:00010000:1.0:1789760797.998734:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28684288 -> 28688384 00010000:00010000:1.0:1789760797.998750:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e075a7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28684288->28688383] (req 28684288->28688383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.998769:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e075a7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28684288->28688383] (req 28684288->28688383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.998778:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e075a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28684288->28688383] (req 28684288->28688383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.998785:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda800 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760797.998789:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e075a7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28684288->28688383] (req 28684288->28688383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.998797:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e075a7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28684288->28688383] (req 28684288->28688383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.998815:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e075a7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28684288->28688383] (req 28684288->28688383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.998823:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041ea3d34/0x3eb3dcc0b2e075a7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28684288->28688383] (req 28684288->28688383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.998911:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4529f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493882880:12345-192.168.203.60@tcp:4:dd.0 Request processed in 47676us (47950us total) trans 4294974322 rc 0/0 00000100:00100000:1.0:1789760797.998923:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7033 00000100:00100000:1.0:1789760797.998933:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808233902055 time 808292256495 state 0x2 ref 0 00000100:00100000:3.0:1789760798.002269:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760798.002550:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493883008 00000100:00100000:0.0:1789760798.002560:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493883008 00000100:00100000:0.0:1789760798.002613:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808233902055 time 808292256495 state 0x0 ref 1 00000100:00100000:0.0:1789760798.002641:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808292516911 time 808292256495 state 0x1 ref 1 00000100:00100000:0.0:1789760798.002670:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808292516911 time 808343705407 state 0x0 ref 1 00000100:00100000:0.0:1789760798.002683:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb452a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493883008:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760798.002735:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28688384, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:0.0:1789760798.002745:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e075ae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.002765:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=808s v=(0 0) 00010000:00010000:0.0:1789760798.002770:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.002775:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e075ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28688384->28692479] (req 28688384->28692479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.002789:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e075ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28688384->28692479] (req 28688384->28692479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.059193:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974323 is committed 00002000:00010000:0.0:1789760798.059358:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28688384 -> 28692480 00010000:00010000:0.0:1789760798.059373:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e075ae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28688384->28692479] (req 28688384->28692479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.059387:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e075ae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28688384->28692479] (req 28688384->28692479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.059396:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e075ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28688384->28692479] (req 28688384->28692479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.059403:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:0.0:1789760798.059408:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e075ae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28688384->28692479] (req 28688384->28692479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.059418:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e075ae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28688384->28692479] (req 28688384->28692479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.059438:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e075ae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28688384->28692479] (req 28688384->28692479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.059447:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e075ae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28688384->28692479] (req 28688384->28692479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.059540:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb452a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493883008:12345-192.168.203.60@tcp:4:dd.0 Request processed in 56854us (57275us total) trans 4294974323 rc 0/0 00000100:00100000:0.0:1789760798.059550:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7034 00000100:00100000:0.0:1789760798.059559:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808292516911 time 808343705407 state 0x2 ref 0 00000100:00100000:2.0:1789760798.062668:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760798.062762:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493883136 00000100:00100000:0.0:1789760798.062771:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493883136 00000100:00100000:0.0:1789760798.062825:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808292516911 time 808343705407 state 0x0 ref 1 00000100:00100000:0.0:1789760798.062840:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808343965823 time 808343705407 state 0x1 ref 1 00000100:00100000:0.0:1789760798.062917:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808343965823 time 808403951924 state 0x0 ref 1 00000100:00100000:0.0:1789760798.062933:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc33e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493883136:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760798.063015:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28692480, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:0.0:1789760798.063025:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e075b5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.063041:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=808s v=(0 0) 00010000:00010000:0.0:1789760798.063046:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.063052:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e075b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28692480->28696575] (req 28692480->28696575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.063079:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e075b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28692480->28696575] (req 28692480->28696575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.121641:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974324 is committed 00002000:00010000:1.0:1789760798.122011:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28692480 -> 28696576 00010000:00010000:1.0:1789760798.122026:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e075b5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28692480->28696575] (req 28692480->28696575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.122039:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e075b5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28692480->28696575] (req 28692480->28696575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.122048:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e075b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28692480->28696575] (req 28692480->28696575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.122063:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afb200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760798.122068:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e075b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28692480->28696575] (req 28692480->28696575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.122077:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e075b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28692480->28696575] (req 28692480->28696575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.122096:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e075b5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28692480->28696575] (req 28692480->28696575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.122104:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df9c6d89/0x3eb3dcc0b2e075b5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28692480->28696575] (req 28692480->28696575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.122336:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc33e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493883136:12345-192.168.203.60@tcp:4:dd.0 Request processed in 59403us (59672us total) trans 4294974324 rc 0/0 00000100:00100000:1.0:1789760798.122348:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7035 00000100:00100000:1.0:1789760798.122358:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808343965823 time 808403951924 state 0x2 ref 0 00000100:00100000:3.0:1789760798.126859:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760798.127067:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493883264 00000100:00100000:0.0:1789760798.127077:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493883264 00000100:00100000:0.0:1789760798.127141:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808343965823 time 808403951924 state 0x0 ref 1 00000100:00100000:0.0:1789760798.127165:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808404212340 time 808403951924 state 0x1 ref 1 00000100:00100000:0.0:1789760798.127197:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808404212340 time 808468233193 state 0x0 ref 1 00000100:00100000:0.0:1789760798.127209:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb453800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493883264:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760798.127259:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28696576, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:0.0:1789760798.127268:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e075bc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.127283:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=808s v=(0 0) 00010000:00010000:0.0:1789760798.127288:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.127295:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e075bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28696576->28700671] (req 28696576->28700671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.127308:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e075bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28696576->28700671] (req 28696576->28700671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.155578:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974325 is committed 00002000:00010000:1.0:1789760798.155770:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28696576 -> 28700672 00010000:00010000:1.0:1789760798.155786:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e075bc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28696576->28700671] (req 28696576->28700671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.155800:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e075bc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28696576->28700671] (req 28696576->28700671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.155809:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e075bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28696576->28700671] (req 28696576->28700671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.155817:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa400 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760798.155822:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e075bc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28696576->28700671] (req 28696576->28700671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.155831:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e075bc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28696576->28700671] (req 28696576->28700671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.155849:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e075bc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28696576->28700671] (req 28696576->28700671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.155856:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025930e8f/0x3eb3dcc0b2e075bc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28696576->28700671] (req 28696576->28700671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.156090:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb453800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493883264:12345-192.168.203.60@tcp:4:dd.0 Request processed in 28877us (29235us total) trans 4294974325 rc 0/0 00000100:00100000:1.0:1789760798.156101:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7036 00000100:00100000:1.0:1789760798.156120:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808404212340 time 808468233193 state 0x2 ref 0 00000100:00100000:1.0:1789760798.159847:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760798.160273:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493883392 00000100:00100000:3.0:1789760798.160285:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493883392 00000100:00100000:3.0:1789760798.160341:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808404212340 time 808468233193 state 0x0 ref 1 00000100:00100000:3.0:1789760798.160367:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808468493609 time 808468233193 state 0x1 ref 1 00000100:00100000:3.0:1789760798.160389:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808468493609 time 808501425090 state 0x0 ref 1 00000100:00100000:3.0:1789760798.160402:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fbb17450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493883392:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760798.160451:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28700672, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:3.0:1789760798.160460:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e075c3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.160475:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=808s v=(0 0) 00010000:00010000:3.0:1789760798.160480:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.160487:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e075c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28700672->28704767] (req 28700672->28704767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.160501:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e075c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28700672->28704767] (req 28700672->28704767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.196901:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974326 is committed 00002000:00010000:3.0:1789760798.197231:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28700672 -> 28704768 00010000:00010000:3.0:1789760798.197248:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e075c3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28700672->28704767] (req 28700672->28704767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.197260:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e075c3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28700672->28704767] (req 28700672->28704767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.197268:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e075c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28700672->28704767] (req 28700672->28704767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.197279:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620f3fda600 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760798.197283:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e075c3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28700672->28704767] (req 28700672->28704767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.197291:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e075c3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28700672->28704767] (req 28700672->28704767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.197309:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e075c3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28700672->28704767] (req 28700672->28704767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.197317:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011c5b9a3/0x3eb3dcc0b2e075c3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28700672->28704767] (req 28700672->28704767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.197450:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fbb17450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493883392:12345-192.168.203.60@tcp:4:dd.0 Request processed in 37046us (37608us total) trans 4294974326 rc 0/0 00000100:00100000:0.0:1789760798.197462:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7037 00000100:00100000:0.0:1789760798.197471:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808468493609 time 808501425090 state 0x2 ref 0 00000100:00100000:3.0:1789760798.200866:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760798.200928:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493883520 00000100:00100000:2.0:1789760798.200937:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493883520 00000100:00100000:2.0:1789760798.201010:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808468493609 time 808501425090 state 0x0 ref 1 00000100:00100000:2.0:1789760798.201024:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808501685506 time 808501425090 state 0x1 ref 1 00000100:00100000:2.0:1789760798.201071:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808501685506 time 808542106064 state 0x0 ref 1 00000100:00100000:2.0:1789760798.201085:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb450000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493883520:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760798.201138:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28704768, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:2.0:1789760798.201147:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075ca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.201162:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=808s v=(0 0) 00010000:00010000:2.0:1789760798.201167:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.201173:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28704768->28708863] (req 28704768->28708863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.201187:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28704768->28708863] (req 28704768->28708863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760798.227481:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28704768 -> 28708864 00010000:00010000:2.0:1789760798.227500:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075ca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28704768->28708863] (req 28704768->28708863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.227514:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075ca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28704768->28708863] (req 28704768->28708863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.227523:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28704768->28708863] (req 28704768->28708863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.227530:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760798.227535:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075ca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28704768->28708863] (req 28704768->28708863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.227543:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075ca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28704768->28708863] (req 28704768->28708863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.227563:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075ca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28704768->28708863] (req 28704768->28708863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.227571:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075ca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28704768->28708863] (req 28704768->28708863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.227664:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb450000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2370:x1876699493883520:12345-192.168.203.60@tcp:4:dd.0 Request processed in 26577us (26799us total) trans 4294974327 rc 0/0 00000100:00100000:2.0:1789760798.227674:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7038 00000100:00100000:2.0:1789760798.227683:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808501685506 time 808542106064 state 0x2 ref 0 00000001:00080000:3.0:1789760798.228135:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974327 is committed 00000100:00100000:3.0:1789760798.231419:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760798.231664:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493883648 00000100:00100000:0.0:1789760798.231674:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493883648 00000100:00100000:0.0:1789760798.231727:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808501685506 time 808542106064 state 0x0 ref 1 00000100:00100000:0.0:1789760798.231743:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808542366480 time 808542106064 state 0x1 ref 1 00000100:00100000:0.0:1789760798.231773:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808542366480 time 808572808833 state 0x0 ref 1 00000100:00100000:0.0:1789760798.231785:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb453480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493883648:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760798.231835:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28708864, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:0.0:1789760798.231844:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e075d1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.231860:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=808s v=(0 0) 00010000:00010000:0.0:1789760798.231864:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.231871:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e075d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28708864->28712959] (req 28708864->28712959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.231884:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e075d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28708864->28712959] (req 28708864->28712959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.261772:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974328 is committed 00002000:00010000:1.0:1789760798.261865:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28708864 -> 28712960 00010000:00010000:1.0:1789760798.261880:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e075d1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28708864->28712959] (req 28708864->28712959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.261894:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e075d1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28708864->28712959] (req 28708864->28712959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.261902:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e075d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28708864->28712959] (req 28708864->28712959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.261910:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa000 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760798.261915:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e075d1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28708864->28712959] (req 28708864->28712959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.261925:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e075d1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28708864->28712959] (req 28708864->28712959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.261948:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e075d1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28708864->28712959] (req 28708864->28712959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.261956:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f01f60ac/0x3eb3dcc0b2e075d1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28708864->28712959] (req 28708864->28712959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.262297:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb453480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2368:x1876699493883648:12345-192.168.203.60@tcp:4:dd.0 Request processed in 30508us (30883us total) trans 4294974328 rc 0/0 00000100:00100000:1.0:1789760798.262310:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7039 00000100:00100000:1.0:1789760798.262321:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808542366480 time 808572808833 state 0x2 ref 0 00010000:00010000:0.0:1789760798.266334:0:5834:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789760798.266676:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760798.267382:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493883776 00000100:00100000:0.0:1789760798.267393:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493883776 00000100:00100000:2.0:1789760798.267512:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808542366480 time 808572808833 state 0x0 ref 1 00000100:00100000:2.0:1789760798.267528:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808573069249 time 808572808833 state 0x1 ref 1 00000100:00100000:2.0:1789760798.267571:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808573069249 time 808608606677 state 0x0 ref 1 00000100:00100000:2.0:1789760798.267585:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc452bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493883776:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760798.267642:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28712960, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:2.0:1789760798.267654:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075d8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.267671:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=808s v=(0 0) 00010000:00010000:2.0:1789760798.267675:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.267682:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28712960->28717055] (req 28712960->28717055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.267826:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28712960->28717055] (req 28712960->28717055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760798.301085:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28712960 -> 28717056 00010000:00010000:1.0:1789760798.301103:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075d8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28712960->28717055] (req 28712960->28717055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.301123:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075d8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28712960->28717055] (req 28712960->28717055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.301133:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28712960->28717055] (req 28712960->28717055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.301140:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760798.301145:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28712960->28717055] (req 28712960->28717055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.301154:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075d8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28712960->28717055] (req 28712960->28717055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.301174:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075d8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28712960->28717055] (req 28712960->28717055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.301182:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075d8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28712960->28717055] (req 28712960->28717055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.301296:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc452bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+7:2369:x1876699493883776:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33708us (34621us total) trans 4294974329 rc 0/0 00000100:00100000:1.0:1789760798.301307:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7040 00000100:00100000:1.0:1789760798.301317:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808573069249 time 808608606677 state 0x2 ref 0 00000001:00080000:3.0:1789760798.301457:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974329 is committed 00000100:00100000:3.0:1789760798.305597:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:0.0:1789760798.305794:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493883904 00000100:00100000:0.0:1789760798.305804:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493883904 00000100:00100000:0.0:1789760798.305867:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808573069249 time 808608606677 state 0x0 ref 1 00000100:00100000:0.0:1789760798.305883:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808608867093 time 808608606677 state 0x1 ref 1 00000100:00100000:0.0:1789760798.305915:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808608867093 time 808646950534 state 0x0 ref 1 00000100:00100000:0.0:1789760798.305927:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb450e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493883904:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:0.0:1789760798.306010:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28717056, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:0.0:1789760798.306020:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e075df lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.306037:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=808s v=(0 0) 00010000:00010000:0.0:1789760798.306042:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.306049:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e075df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28717056->28721151] (req 28717056->28721151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.306209:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e075df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28717056->28721151] (req 28717056->28721151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.346551:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974330 is committed 00002000:00010000:3.0:1789760798.346715:0:6694:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28717056 -> 28721152 00010000:00010000:3.0:1789760798.346729:0:6694:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e075df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28717056->28721151] (req 28717056->28721151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.346741:0:6694:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e075df lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28717056->28721151] (req 28717056->28721151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.346756:0:6694:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e075df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28717056->28721151] (req 28717056->28721151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.346764:0:6694:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff961fc4afa200 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:3.0:1789760798.346770:0:6694:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e075df lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28717056->28721151] (req 28717056->28721151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.346779:0:6694:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e075df lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28717056->28721151] (req 28717056->28721151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.346798:0:6694:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e075df lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28717056->28721151] (req 28717056->28721151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.346806:0:6694:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000200af3ae/0x3eb3dcc0b2e075df lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28717056->28721151] (req 28717056->28721151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.346896:0:6694:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb450e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493883904:12345-192.168.203.60@tcp:4:dd.0 Request processed in 40966us (41303us total) trans 4294974330 rc 0/0 00000100:00100000:3.0:1789760798.346906:0:6694:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7041 00000100:00100000:3.0:1789760798.346916:0:6694:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808608867093 time 808646950534 state 0x2 ref 0 00000100:00100000:0.0:1789760798.351449:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760798.351547:0:6692:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493884032 00000100:00100000:2.0:1789760798.351557:0:6692:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493884032 00000100:00100000:2.0:1789760798.351612:0:6692:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808608867093 time 808646950534 state 0x0 ref 1 00000100:00100000:2.0:1789760798.351627:0:6692:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808647210950 time 808646950534 state 0x1 ref 1 00000100:00100000:2.0:1789760798.352370:0:6692:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808647210950 time 808693401678 state 0x0 ref 1 00000100:00100000:2.0:1789760798.352388:0:6692:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493884032:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760798.352440:0:6692:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28721152, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:2.0:1789760798.352449:0:6692:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075e6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.352464:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=808s v=(0 0) 00010000:00010000:2.0:1789760798.352468:0:6692:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.352474:0:6692:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28721152->28725247] (req 28721152->28725247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.352487:0:6692:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28721152->28725247] (req 28721152->28725247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.385292:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974331 is committed 00002000:00010000:2.0:1789760798.385448:0:6692:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28721152 -> 28725248 00010000:00010000:2.0:1789760798.385463:0:6692:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28721152->28725247] (req 28721152->28725247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.385475:0:6692:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075e6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28721152->28725247] (req 28721152->28725247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.385485:0:6692:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28721152->28725247] (req 28721152->28725247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.385493:0:6692:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:2.0:1789760798.385498:0:6692:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28721152->28725247] (req 28721152->28725247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.385507:0:6692:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075e6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28721152->28725247] (req 28721152->28725247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.385631:0:6692:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075e6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28721152->28725247] (req 28721152->28725247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.385640:0:6692:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075e6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28721152->28725247] (req 28721152->28725247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6692 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.386074:0:6692:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb460a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2369:x1876699493884032:12345-192.168.203.60@tcp:4:dd.0 Request processed in 33685us (34630us total) trans 4294974331 rc 0/0 00000100:00100000:2.0:1789760798.386090:0:6692:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7042 00000100:00100000:2.0:1789760798.386101:0:6692:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808647210950 time 808693401678 state 0x2 ref 0 00000100:00100000:1.0:1789760798.392651:0:3297:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:2.0:1789760798.392746:0:13038:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493884160 00000100:00100000:2.0:1789760798.392755:0:13038:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493884160 00000100:00100000:2.0:1789760798.392810:0:13038:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808647210950 time 808693401678 state 0x0 ref 1 00000100:00100000:2.0:1789760798.392825:0:13038:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808693662094 time 808693401678 state 0x1 ref 1 00000100:00100000:2.0:1789760798.392855:0:13038:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808693662094 time 808733890715 state 0x0 ref 1 00000100:00100000:2.0:1789760798.392868:0:13038:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4528a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493884160:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:2.0:1789760798.392918:0:13038:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28725248, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:2.0:1789760798.392927:0:13038:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075ed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.392943:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=808s v=(0 0) 00010000:00010000:2.0:1789760798.392948:0:13038:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.392954:0:13038:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28725248->28729343] (req 28725248->28729343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.393009:0:13038:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28725248->28729343] (req 28725248->28729343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.438900:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974332 is committed 00002000:00010000:1.0:1789760798.439019:0:13038:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28725248 -> 28729344 00010000:00010000:1.0:1789760798.439035:0:13038:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28725248->28729343] (req 28725248->28729343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.439047:0:13038:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075ed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28725248->28729343] (req 28725248->28729343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.439068:0:13038:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28725248->28729343] (req 28725248->28729343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.439076:0:13038:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff96210193ce00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760798.439081:0:13038:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28725248->28729343] (req 28725248->28729343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.439091:0:13038:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075ed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28725248->28729343] (req 28725248->28729343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.439239:0:13038:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075ed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28725248->28729343] (req 28725248->28729343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.439249:0:13038:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076dd0a0f/0x3eb3dcc0b2e075ed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28725248->28729343] (req 28725248->28729343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13038 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.439376:0:13038:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff961fc4528a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493884160:12345-192.168.203.60@tcp:4:dd.0 Request processed in 46505us (46728us total) trans 4294974332 rc 0/0 00000100:00100000:1.0:1789760798.439387:0:13038:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7043 00000100:00100000:1.0:1789760798.439397:0:13038:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808693662094 time 808733890715 state 0x2 ref 0 00000100:00100000:3.0:1789760798.443666:0:3298:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:1.0:1789760798.444081:0:6693:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493884288 00000100:00100000:1.0:1789760798.444091:0:6693:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493884288 00000100:00100000:1.0:1789760798.444330:0:6693:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808693662094 time 808733890715 state 0x0 ref 1 00000100:00100000:1.0:1789760798.444347:0:6693:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808734151131 time 808733890715 state 0x1 ref 1 00000100:00100000:1.0:1789760798.444486:0:6693:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808734151131 time 808785520743 state 0x0 ref 1 00000100:00100000:1.0:1789760798.444501:0:6693:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9620fb452a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493884288:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:1.0:1789760798.444560:0:6693:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28729344, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:1.0:1789760798.444569:0:6693:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e075f4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.444584:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=808s v=(0 0) 00010000:00010000:1.0:1789760798.444589:0:6693:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.444595:0:6693:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e075f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28729344->28733439] (req 28729344->28733439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.444609:0:6693:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e075f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28729344->28733439] (req 28729344->28733439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.489878:0:6716:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294974333 is committed 00002000:00010000:1.0:1789760798.490066:0:6693:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 28729344 -> 28733440 00010000:00010000:1.0:1789760798.490082:0:6693:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e075f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28729344->28733439] (req 28729344->28733439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.490096:0:6693:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e075f4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28729344->28733439] (req 28729344->28733439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.490224:0:6693:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e075f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28729344->28733439] (req 28729344->28733439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.490233:0:6693:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9620c1495a00 already unused, calling callback (ffffffffc17d3350) 00010000:00010000:1.0:1789760798.490237:0:6693:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e075f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28729344->28733439] (req 28729344->28733439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.490247:0:6693:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e075f4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28729344->28733439] (req 28729344->28733439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.490268:0:6693:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e075f4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28729344->28733439] (req 28729344->28733439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.490275:0:6693:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090e29942/0x3eb3dcc0b2e075f4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28729344->28733439] (req 28729344->28733439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6693 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.490407:0:6693:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9620fb452a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2367:x1876699493884288:12345-192.168.203.60@tcp:4:dd.0 Request processed in 45903us (46744us total) trans 4294974333 rc 0/0 00000100:00100000:1.0:1789760798.490418:0:6693:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.60@tcp, seq: 7044 00000100:00100000:1.0:1789760798.490428:0:6693:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808734151131 time 808785520743 state 0x2 ref 0 00000100:00100000:0.0:1789760798.494506:0:3296:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.60@tcp (source: 12345-192.168.203.60@tcp) 00000100:00100000:3.0:1789760798.494686:0:6694:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699493884416 00000100:00100000:3.0:1789760798.494695:0:6694:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699493884416 00000100:00100000:3.0:1789760798.494747:0:6694:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808734151131 time 808785520743 state 0x0 ref 1 00000100:00100000:3.0:1789760798.494765:0:6694:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808785781159 time 808785520743 state 0x1 ref 1 00000100:00100000:3.0:1789760798.494799:0:6694:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff9620fae9a800 key nid={192.168.203.60@tcp}&opcode={ost_write} rule@0xffff9620ccad7400 gen 0 rate 15 depth 3 token 3 deadline 808785781159 time 808835834167 state 0x0 ref 1 00000100:00100000:3.0:1789760798.494811:0:6694:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff961fc4d88e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:9b240726-48d9-467b-83a8-5b29f6d3cb3b+5:2370:x1876699493884416:12345-192.168.203.60@tcp:4:dd.0 00002000:00010000:3.0:1789760798.494860:0:6694:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 28733440, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xe009 00010000:00010000:3.0:1789760798.494870:0:6694:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e075fb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.494885:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=808s v=(0 0) 00010000:00010000:3.0:1789760798.494890:0:6694:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.494895:0:6694:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e075fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28733440->28737535] (req 28733440->28737535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.494910:0:6694:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000069f3c2b6/0x3eb3dcc0b2e075fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [28733440->28737535] (req 28733440->28737535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6694 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 Debug log: 56392 lines, 56392 kept, 0 dropped, 0 bad.